Reactions of EFL Learners to an Unfairness Test

Hossein  Salarian

doi:10.61227/jetti.v3i1.174

Authors

Hossein Salarian University of Tehran, Tehran, Iran

DOI:

https://doi.org/10.61227/jetti.v3i1.174

Keywords:

Bias, fairness, unfairness, ethical test preparation, social class

Abstract

The matter of bias is more muddled by the problem of obviously differentiating components of culture and instructional background from the language capacities we want to measure. This study investigates the reaction of English as a Foreign Language (EFL) learners to perceived test unfairness, examining the role of social class and gender in test performance. Drawing on fairness theories and using the reading section of the IELTS test, thirty-four intermediate-level learners were initially pretested and grouped according to gender and social class. After exposure to a uniform instructional treatment, participants completed a posttest. For this purpose, both descriptive and inferential statistics were used to compare the results of the groups. In order to make sure that the differences between the groups are not statistically significant, a one-way ANOVA was employed. Results revealed significant disparities in performance linked to social class and gender, suggesting the presence of test unfairness or differential access to the skills required. This study underscores the importance of validating assessment instruments not only in terms of construct validity but also in terms of fairness across sociocultural demographics. There are some theoretical and practical implications involved in this study.

Downloads

Download data is not yet available.

References

Ahmadi Safa, M. & Nasiri, B. (2025).Fairness in language classroom assessment practices: what do EFL teachers underscore?. Language Testing in Asia, 15(1).49-62. Article number: 1 (2025)

AERA (1999). American Educational Research Association, American Psychological Association and National Council on Measurement in Education. Standards for educational and psycho¬logical testing. Washington, DC.

Bachman, L. F. (2005). Building a test use argument. Language Assessment Quarterly, 2, 1–34.

Backman, L. F. (2010). Fundamental considerations in language testing. Oxford: Oxford University Press.

Boyd Jenkins, O. (2003). The meaning of “fairness”. The Virtual Research Centre: Links for Learning. http://www.orvillejenkins.com/faithlife/fairnessfl.html

Childs, R. A. (1990). Gender bias and fairness. ERIC Digest, ED328610. http://www.ericdigests,org/pre-9218/gender.htm

Camilli, G. (2006). Test fairness. In R. Brennan (Ed.), Educational measurement, 4th ed. (pp. 221–256), Westport, CT: American Council on Education and Praeger.

Chapelle, C. A., Enright, M. K. & Jamieson, J. M. (2008). (Eds.), Building a validity argument for the Test of English as a Foreign Language. Mahwah, NJ: Lawrence Erlbaum.

Elder, C. (1997). What does test bias have to do with fairness? Language Testing, 14(3), 261– 277.

Herman, J., & Cook, L. (2019). Fairness in classroom assessment. In S. M. Brookhart & J. H. McMillan (Eds.), Classroom assessment and educational measurement (pp. 243–264). Routledge.

Holcombe, R. G. (1983). Applied fairness theory: comment. The American Economic Review, 73(5), 1153–1156.

Gao, S. & Liu, J. (2023). Ethical issues in language testing from a validity perspective. Open Journal of Social Sciences, 11(4), 143-158.

Gipps (1990), beyond testing. Harvard University Press.

Jones, K., Evans, C., Byrd, R., Campbell, K. (2000) Gender equity training and teaching behavior. Journal of Instructional Psychology, 27(3), 173-178.

Kane, M. T. (2006). Validation. In Brennan, R. L. (Ed.), Educational measurement, 4th ed.(pp. 18–64). Washington, DC: American Council on Education/Praeger.

Kunnan, A. J. (1997). Connecting validation and fairness in language testing. In A. Huhta et al. (Ed.), Current developments and alternatives in language assessment (pp. 85–105). Jyväskylä, Finland: University of Jyväskylä.

Kunnan, A. J. (2000). Fairness and justice for all. In A. J. Kunnan (Ed.), Fairness and validation in language assessment (pp. 1–14). Cambridge, UK: Cambridge University Press.

Kunnan, A. J. (2008). Towards a model of test evaluation: Using the Test Fairness and Wider Con¬text frameworks. In L. Taylor & C. Weir (Eds.), Multilingualism and assessment: Achieving transparency, assuring quality, sustaining diversity (pp. 229–251). Cambridge, UK: Cambridge University Press.

Kunnan, A. J. (2010). Introduction: Test fairness, test bias and DIF. Language Assessment Quar¬terly, 4(2), 109–112.

Kunnan, A. J. (2004). Test fairness. In Milanovic, M. & Weir C., (Eds.), European language testing in a global context: Proceedings of the ALTE Barcelona Conference (pp. 27–48). Cambridge, UK: Cambridge University Press.

Kunnan, A. J. (2000). Fairness and justice for all. In A. J. Kunnan (Ed.), Fairness and validation in language assessment (pp. 1–14). Cambridge, UK: Cambridge University Press.

Lam, T. C. M. (1995). Fairness in performance assessment. ERIC Digest, ED391982. http://www.ericdigests.org/1996–4/fairness.htm

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement, 3rd ed. (pp. 13– 103). New York: American Council on Education and Macmillan.

Mislevy, R. J. (2018). Sociocognitive foundations of educational measurement. Routledge.

Mazzoli Smith, L., Todd, L., & Laing, K. (2018). Students’ views on fairness in education: The importance of relational justice and stakes fairness. Research Papers in Education, 33, 336–353.

Murillo, F. J., & Hidalgo, N. (2020). Fair student assessment: A phenomenographic study on teachers’ conceptions. Studies in Educational Evaluation, 65, 1–10.

Nisbet, I. (2019). Fairness takes center stage. Assessment in Education Principles Policy and Practice, 26(1), 111–117.

Peters, R., Kruse, J., Buckmiller, T., & Townsley, M. (2017). It’s just not fair! Making sense of secondary students’ resistance to a standards-based grading. American Secondary Education, 45(3), 9–27.

Popham, W. J. (1997). Consequential validity: Right concern – wrong concept. Educational Measure¬ment: Issues and Practice, 16(2), 9–13.

Porter, T. (2003). Measurement, objectivity, and trust. Measurement: Interdisciplinary Research and Perspectives, 1, 241–255.

Rawls, J. (2001). Justice as Fairness: A restatement. Cambridge, MA: Harvard University Press.

Sadker, D., Sadker, M. (1994). Failing at Fairness: How Our Schools Cheat Girls. Toronto, ON: Simon & Schuster Inc.

Saunders, A. (2008). Egalitarianism and fairness. The Philosopher’s Zone. ABS Radio. June 2008. http://www.abc.net.au/rn/philosopherszone/stories/2008/2275460.htm

Shohamy, E. (2000). Fairness in language testing. In Kunnan, A. J. (Ed.), Fairness and validation in language assessment (pp. 15–19). Cambridge, UK: Cambridge University Press.

Velasquez, M., Andre, C., Shanks, T., & Meyer, M. J. (2008). Justice and fairness. Markkula Center for Applied Ethics. Santa Clara University, California. http://www.scu.edu/ethics/practicing/decision/ justics.html

Willingham, W. W. & N. Cole (1997). Gender and fair assessment. Mahwah, NJ: Lawrence Erlbaum.

Willingham, W. W. (1999). A systemic view of test fairness. In Messick S. (Ed.), Assessment in higher education: Issues in access, quality, student development, and public policy(pp. 213–242). Mahwah, NJ: Lawrence Erlbaum.

Zhao, M. R., Mu, B. L., & Lu, C. P. (2016). Teaching to the test: approaches to teaching in senior secondary schools in the context of curriculum reform in China. Creative Education, 7, 32- 43.

Zeidner, M. (1986). Are English language aptitude tests biased towards culturally different minor¬ity groups? Some Israeli findings. Language Testing, 3, 80–95.

Zeidner, M. (1987). A comparison of ethnic, sex and age biases in the predictive validity of English language aptitude tests: Some Israeli data. Language Testing, 4, 55–71.