EFFECTS OF IDIOMS ON TEACHER RATINGS OF L2 WRITING QUALITY AND LEXICAL DIVERSITY, AND ASSESSMENT CHALLENGES

Eihab Abu-Rabiah

doi:10.24071/llt.v28i2.12112

EFFECTS OF IDIOMS ON TEACHER RATINGS OF L2 WRITING QUALITY AND LEXICAL DIVERSITY, AND ASSESSMENT CHALLENGES

Eihab Abu-Rabiah^(1*),

(1) Kaye Academic College of Education and Ben-Gurion University of the Negev, Israel
(*) Corresponding Author

Abstract

Idioms are often seen as signs of advanced language proficiency, yet their role in writing assessment is given little consideration. Most research has focused on how idioms are processed cognitively, leaving open questions about how they affect teacher judgments, particularly in second language writing. This study explores how idioms affect teacher ratings of lexical diversity and writing quality, while also highlighting the challenges teachers face in evaluating them. Using a mixed-methods design, twenty Hebrew teachers assessed fifteen essays written by native Arabic speakers, along with revised versions of each essay that included a frequent idiom (30 essays total). The essays spanned CEFR levels A2 to B2 and were randomly ordered. Teachers, whether native speakers of Arabic or Hebrew, rated both lexical diversity and writing quality of each essay on a 10-point scale. Six teachers also participated in semi-structured interviews, which were subjected to thematic analysis. The findings indicate that essays with idioms were rated significantly higher in both writing quality and lexical diversity, regardless of the rater’s first language. The interviews highlighted several recurring challenges in assessing idioms, including contextual appropriateness, idiom transparency, frequency of use, grammatical accuracy, and variation between standard and colloquial forms.

Keywords

formulaic language; Hebrew as a second language; L2 writing quality assessment; lexical diversity; teacher rating

Full Text:

PDF

References

Abu-Gweder, A. (2023). The challenges of writing among Arab-Bedouin students between literary Arabic and Hebrew as a second language. Journal of Namibian Studies: History Politics Culture, 34, 769–787. https://doi.org/10.59670/jns.v34i.1124

Abu-Rabiah, E. (2020). Lexical measures for testing progress in Hebrew as Arab students’ L2. Journal of Language and Linguistic Studies, 16(3), 1096–1114. https://doi.org/10.17263/jlls.803551

Abu-Rabiah, E. (2023). Evaluating L2 vocabulary development features using lexical density and lexical diversity measures. LLT Journal: A Journal on Language and Language Teaching, 26(1), 168–182. https://doi.org/10.24071/llt.v26i1.5841

Abu-Rabiah, E. (2024). Exploring lexical sophistication in second language: An analysis of vocabulary using a word-rating method. Theory and Practice of Second Language Acquisition, 2(10), 1–24. https://doi.org/10.31261/TAPSLA.15623

Abu-Rabiah, E. (2025a). Semantic and phonological effects on L2 to L1 translation: The case of cognates, false cognates, and translation equivalents. Electronic Journal of Foreign Language Teaching, 22(1), 25–40. https://doi.org/10.56040/eiar2212

Abu-Rabiah, E. (2025b). The productive vocabulary size of second language learners upon entry into higher education. Theory and Practice of Second Language Acquisition, 11(1), 1–19. https://doi.org/10.31261/TAPSLA.16594

Abu-Rabiah, E. (2025c). Navigating language barriers: Hebrew proficiency and self-efficacy among Muslim Arab Bedouin students in Israeli higher education. Journal of International Students, 15(3), 71–90. https://doi.org/10.32674/s08gad96

Abu-Rabiah, E., Gafter, R., & Henkin, R. (2023). Where syntactic interference persists: The case of Hebrew written by native Arabic speakers. L1-Educational Studies in Language and Literature, 23, 1–19. https://doi.org/10.21248/l1esll.2023.23.1.399

Adams, W. C. (2015). Conducting semi-structured interviews. In K. E. Newcomer, H. P. Hatry, J. S. Wholey, K. E. Newcomer, H. P. Hatry, & J. S. Wholey (Eds.), Handbook of practical program evaluation (4th ed.) (pp. 492–505). John Wiley & Sons, Inc. https://doi.org/10.1002/9781119171386.ch19

Al-kadi, A. M. T. (2015). Towards idiomatic competence of Yemeni EFL undergraduates. Journal of Language Teaching and Research, 6(3), 513–523. http://dx.doi.org/10.17507/jltr.0603.06

Anjarini, R. D., & Hatmanto, E. D. (2021, December). Challenges and strategies in understanding English idioms: English as a foreign language students’ perception. In International Conference on Sustainable Innovation Track Humanities Education and Social Sciences (ICSIHESS 2021) (pp. 241–247). Atlantis Press. http://dx.doi.org/10.2991/assehr.k.211227.039

Atai, M. R., & Akbarian, I. (2003). The effect of exposure on EFL learners’ acquisition of idioms with reference to proficiency levels. Indian Journal of Applied Linguistics, 29(1), 21–34.

Bestgen, Y., & Granger, S. (2014). Quantifying the development of phraseological competence in L2 English writing: An automated approach. Journal of Second Language Writing, 26, 28–41. https://doi.org/10.1016/j.jslw.2014.09.004

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Cacciari, C. (2014). Processing multiword idiomatic strings: Many words in one?. The Mental Lexicon, 9(2), 267–293. https://doi.org/10.1075/ml.9.2.05cac

Cacciari, C., & Tabossi, P. (2014). Idioms: Processing, structure, and interpretation. Psychology Press.

Carrol, G., & Conklin, K. (2020). Is all formulaic language created equal? Unpacking the processing advantage for different types of formulaic sequences. Language and Speech, 36(1), 95–122. https://doi.org/10.1177/0023830918823230

Casas, R. and Campoy, J. (1995). A sociolinguistic approach to the study of idioms: Some anthropolinguistic sketches. Cuadernos de Filología Inglesa, 4, 43–61.

Crossley, S. A. (2020). Linguistic features in writing quality and development: An overview. Journal of Writing Research, 11(3), 415–443. https://doi.org/10.17239/jowr-2020.11.03.01

Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35(2), 115–135. https://doi.org/10.1111/j.1467-9817.2010.01449.x

Ellis, N. C., Simpson‐Vlach, R. I. T. A., & Maynard, C. (2008). Formulaic language in native and second language speakers: Psycholinguistics, corpus linguistics, and TESOL. TESOL Quarterly, 42(3), 375–396. https://doi.org/10.1002/j.1545-7249.2008.tb00137.x

Espinal, M. T., & Mateu, J. (2019). Idioms and phraseology. In M. T. Espinal & J. Mateu (Eds.), Oxford research encyclopedia of linguistics. Oxford University Press. https://doi.org/10.1093/acrefore/9780199384655.013.51

Gramley, S., Gramley, V., & Pätzold, K. M. (2020). A survey of modern English. Routledge. https://doi.org/10.4324/9780429300356

Hajiyeva, B. (2024). Challenges in understanding idiomatic expressions. Acta Globalis Humanitatis et Linguarum, 1(2), 67–73. https://doi.org/10.69760/aghel.01024066

Haskel-Shaham, I., Klaus, R., & Tamir, R. (2018). Discourse―yes, grammar― no. Influence of Arabic mother tongue on Arab students writing in Hebrew. L1-Educational Studies in Language and Literature, 18, 1–26. https://doi.org/10.17239/L1ESLL-2018.18.01.07

Henkin, R. (2020). Persistence of interference from L1 Arabic in written Hebrew. L1-Educational Studies in Language and Literature, 20, 1–31. https://doi.org/10.17239/L1ESLL-2020.20.01.15

Howarth, P. (1998). The phraseology of learners’ academic writing. In A. Cowie (Ed.), Phraseology: Theory, analysis, and applications (pp. 161–186). Oxford University Press. https://doi.org/10.1093/oso/9780198294252.003.008

Hubers, F., Cucchiarini, C., Strik, H., & Dijkstra, T. (2019). Normative data of Dutch idiomatic expressions: Subjective judgments you can bank on. Frontiers in Psychology, 10, 1075. https://doi.org/10.3389/fpsyg.2019.01075

Jarvis, S. (2017). Grounding lexical diversity in human judgments. Language Testing, 34(4), 537–553. https://doi.org/10.1177/0265532217710632

Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133. https://doi.org/10.1177/1558689806298224

Joshi, A., Kale, S., Chandel, S., & Pal, D. K. (2015). Likert scale: Explored and explained. British Journal of Applied Science & Technology, 7(4), 396–403. https://doi.org/10.9734/BJAST/2015/14975

Kashiha, H., & Chan, S. H. (2015). A little bit about: Differences in native and non-native speakers’ use of formulaic language. Australian Journal of Linguistics, 35(4), 297–310. https://doi.org/10.1080/07268602.2015.1067132

Kiger, M. E., & Varpio, L. (2020). Thematic analysis of qualitative data: AMEE Guide No. 131. Medical Teacher, 42(8), 846–854. https://doi.org/10.1080/0142159X.2020.1755030

Le Sourn-Bissaoui, S., Caillies, S., Bernard, S., Deleau, M., & Brulé, L. (2012). Children’s understanding of ambiguous idioms and conversational perspective-taking. Journal of Experimental Child Psychology, 112(4), 437–451. https://doi.org/10.1016/j.jecp.2012.02.003

Lewis, M. (1993). The lexical approach: The state of ELT and a way forward. Language Teaching Publications.

Li, L. (2023). Quadri-syllabic schematic idioms in Chinese: Description and acquisition. Springer. https://doi.org/10.1007/978-981-19-7202-7

Listyani, L., & Thren, A. T. (2023). Indonesian EFL learners’ errors in idiomatic expressions: Teachers’ and students’ perspectives. European Journal of Education Studies, 10(10), 374–392. http://dx.doi.org/10.46827/ejes.v10i10.5036

Liu, D. (2017). Idioms: Description, comprehension, acquisition, and pedagogy. Routledge. https://doi.org/10.4324/9781315092843

Liu, L., & Cheung, H. T. (2014). Acquisition of Chinese quadra-syllabic idiomatic expressions: Effects of semantic opacity and structural symmetry. First Language, 34(4), 336–353. https://doi.org/10.1177/0142723714544409

Liu, Y., Pang, B., & Liu, B. (2019). Neural-based Chinese idiom recommendation for enhancing elegance in essay writing. Annual Meeting of the Association for Computational Linguistics.

Liu, Z. (2012). Analysis of idiom variation in the framework of linguistic subjectivity. English Language Teaching, 5(6), 105–113.

Malvern, D., Richards, B., Chipere, N., and Durán, P. (2004). Lexical diversity and language development: Quantification and assessment. Palgrave Macmillan. https://doi.org/10.1057/9780230511804

McCarthy, P. M., & Jarvis, S. (2007). Vocd: A theoretical and empirical evaluation. Language Testing, 24(4), 459–488. https://doi.org/10.1177/0265532207080767

McGlone, M. S., Glucksberg, S., & Cacciari, C. (1994). Semantic productivity and idiom comprehension. Discourse Processes, 17(2), 167–190. https://doi.org/10.1080/01638539409544865

Meara, P. (2014). Review of the book Vocabulary knowledge: Human ratings and automated measures, by Scott Jarvis and Michael Daller (Eds.). International Journal of Applied Linguistics, 24(3), 418–421. https://doi.org/10.1111/ijal.12086

Mehdi, S. (2024). Techniques used in teaching idioms. Acta Globalis Humanitatis et Linguarum, 1(2), 34–42. https://doi.org/10.69760/aghel.01024063

Milburn, E., Vulchanova, M., & Vulchanov, V. (2021). Collocational frequency and context effects on idiom processing in advanced L2 speakers. Canadian Journal of Experimental Psychology, 75(2), 169–174. https://doi.org/10.1037/cep0000214

Miller, J. (2020). The bottom line: Are idioms used in English academic speech and writing? Journal of English for Academic Purposes, 43, Article 100810. https://doi.org/10.1016/j.jeap.2019.100810

O’Grady, S. (2023). Halo effects in rating data: Assessing speech fluency. Research Methods in Applied Linguistics, 2(2), Article 100048. https://doi.org/10.1016/j.rmal.2023.100048

O’Loughlin, K. J. (1994). The assessment of writing by English and ESL teachers. Australian Review of Applied Linguistics, 17(1), 23–44. https://doi.org/10.1075/aral.17.1.02olo

Paquot, M. (2018). Phraseological competence: A missing component in university entrance language tests? Insights from a study of EFL learners’ use of statistical collocations. Language Assessment Quarterly, 15(1), 29–43. https://doi.org/10.1080/15434303.2017.1405421

Park, J., & Chon, Y. V. (2019). EFL learners’ knowledge of high-frequency words in the comprehension of idioms: A boost or a burden? RELC journal, 50(2), 219–234. https://doi.org/10.1177/00336882177480

Plano Clark, V. L. (2017). Mixed methods research. The Journal of Positive Psychology, 12(3), 305–306. https://doi.org/10.1080/17439760.2016.1262619

Qadr, H. M., & Madhat, I. R. (2020). The relationship between idiomatic knowledge and second language proficiency/Kurdistan universities undergraduates as an example. International Journal of Linguistics, Literature and Translation, 3(8), 149–160. https://al-kindipublisher.com/index.php/ijllt/article/view/374

Rafatbakhsh, E., & Ahmadi, A. (2019). A thematic corpus-based study of idioms in the Corpus of Contemporary American English. Asian-Pacific Journal of Second and Foreign Language Education, 4(1), Article 11. https://doi.org/10.1186/s40862-019-0076-4

Saberian, N., & Fotovatnia, Z. (2011). Idiom taxonomies and idiom comprehension: Implications for English teachers. Theory and Practice in Language Studies, 1(9), 1231–1235.

Senaldi, M. S. G., & Titone, D. (2024). Idiom meaning selection following a prior context: Eye movement evidence of L1 direct retrieval and L2 compositional assembly. Discourse Processes, 61(1–2), 21–43. https://doi.org/10.1080/0163853X.2024.2311637

Shi, L. (2001). Native- and nonnative-speaking EFL teachers’ evaluation of Chinese students’ English writing. Language Testing, 18(3), 303–325. https://doi.org/10.1177/026553220101800303

Sinclair, J. (1991). Corpus, concordance, collocation. Oxford University Press

Siyanova-Chanturia, A., Conklin, K., & Schmitt, N. (2011). Adding more fuel to the fire: An eye-tracking study of idiom processing by native and non-native speakers. Second Language Research, 27(2), 251–272. https://doi.org/10.1177/0267658310382068

Suhodolli, M., & Lama, A. (2024). Teaching English idioms: A focus on cultural adaptation and contextual learning in Kosovo—Case study: Public international business college in Mitrovica, Kosovo). Journal of Educational and Social Research, 14(6), 585–595. https://doi.org/10.36941/jesr-2024-0195

Suñer, F. (2019). Exploring the relationship between stay-abroad experiences, frequency effects, and context use in L2 idiom comprehension. Journal of Language Teaching and Research, 10(1), 35–44. http://dx.doi.org/10.17507/jltr.1001.04

Ta’amneh, M. A. A. A. (2021). Strategies and difficulties of learning English idioms among university students. Strategies, 12(23), 76–84.

Tiv, M., Milburn, E., & Warren, T. (2016). Investigating the effects of transparency and ambiguity on idiom learning. Annual Meeting of the Cognitive Science Society.

Türker, E. (2018). The influence of L1 frequency in instructed second language learning of L2 idioms. Journal of Second Language Studies, 1(2), 333–356. https://doi.org/10.1075/jsls.17007.tur

Vanderniet, K. H. (2015). Idioms as a measure of proficiency. Brigham Young University.

Vasiljevic, Z. (2015). Teaching and learning idioms in L2: From theory to practice. MAXTESOL Journal, 39(4), 1–24.

Vögelin, C., Jansen, T., Keller, S. D., & Möller, J. (2021). The impact of vocabulary and spelling on judgments of ESL essays: An analysis of teacher comments. The Language Learning Journal, 49(6), 631–647. https://doi.org/10.1080/09571736.2018.1522662

Wagner, W. (2020). Idioms and ambiguity in context: Phrasal and compositional readings of idiomatic expressions. Mouton De Gruyter. https://doi.org/10.1515/9783110685459

Zhang, Y., & Elder, C. (2011). Judgments of oral proficiency by non-native and native English-speaking teacher raters: Competing or complementary constructs? Language Testing, 28(1), 31–50. https://doi.org/10.1177/0265532209360671

DOI: https://doi.org/10.24071/llt.v28i2.12112