THE EFFECTS OF RATING EXPERIENCE AND DISCOURSE MODES ON ASSESSING EFL LEARNERS WRITING PERFORMANCES

Rania Zribi, Chokri Smaoui

Abstract


This study aims at investigating the effects of discourse modes on assessing EFL learners written performances. A total of fifty raters judged sixty essays (30 narratives and 30 argumentative writing modes) written by third-year English students from the Faculty of Letters and Humanities. Raters not only scored the compositions but also justified their scores assignments based on written explanations. Raters rating behaviors were diagnosed based on a variety of quantitative and qualitative tools. Essay scores were analyzed based on the statistical model FACETS to measure raters severity and internal consistency, task difficulty, and the scale functioning across writing modes. Qualitative data (gathered from interviews and report forms) were also analyzed in order to examine which aspects of writing were deemed more important than others across task types. The analysis revealed that the discourse mode was substantially an influential factor. The narrative task was more difficult than the argumentative one. Narrative essays were judged harsher than argumentative essays. Less consistent ratings could be detected from the narrative mode, compared to the argumentative one. Qualitative findings showed that the two writing modes were different in their qualitative judgments due to their different genre requirements and norms.


Keywords



Full Text:

PDF

References


Barkaoui, K. (2008). Effects of scoring method and rater experience on ESL essay rating processes and outcomes. Unpublished doctoral dissertation. University of Toronto, Canada.

Bizzell, P. (1987). Review: What can we know, what must we do, what may we hope: writing assessment. College English, vol. 49, (5), pp. 575-584.

Carrell, P., L. (1995). The Effect of Writers Personalities and Raters Personalities on the Holistic Evaluation of Writing. Assessing Writing, vol. 2, (2), pp. 153-l90.

Collier, D. (1993). The comparative method. In A. W. Finifter (Ed.). Political Science: The State of the Discipline 2. American Political Science Association.

Cumming, A., Kantor, R. & Powers, D., E. (2002). Decision making while rating ESL/EFL writing tasks: A descriptive framework. Modern Language Journal, vol.86, (1), pp. 67-96.

Engelhard Jr, G., Gordon, B., & Gabrielson, S. (1991). Writing Tasks and the Quality of Student Writing: Evidence from a Statewide Assessment of Writing. Paper presented at the Annual Meeting of the American Educational Research Association.

Engelhard, G., Gordon, B., & Gabrielson, S. (1992). The influences of mode of discourse, experiential demand, and gender on the quality of student writing. Research in the Teaching of English, vol. 26, pp. 315-336.

Huot, B. (1990). The literature of direct writing assessment: major concerns and prevailing trends. Review of Educational Research Summer, vol. 60, (2), pp. 237-263.

Jacob, H., Zinkgraf, S., Wormuth, D., Hartfiel, V. F. & Hughey, J. (1981). Testing EFL Composition: A Practical Approach. Rowley, Mass: Newbury House.

Kegley, P., H. (1986). The effect of mode discourse on student writing performance: Implications for policy. Educational Evaluation and Policy Analysis, vol. 8, (2), pp. 147-154.

Kuhlemeier, H., van den Bergh, H., & Wijnstra, J. (1995). Multilevel factor analysis applied to national assessment data. Paper presented at the Annual meeting of the American Educational Research Association, San Francisco.

Oxford, R., (1996). When emotion meets (meta) cognition in language learning histories. The Teaching of Culture and Language in the Second Language Classroom, pp. 581-594.

Quellmalz, E., Capell, F, & Chou, C., P. (1982). Effects of discourse and response mode on the measurement of writing competence. Journal of Educational Measurement, vol.19, pp. 241-258.

Quellmalz, E., Capell, F., J., & Chou, C., P. (1980). Defining writing: Effects of discourse and response mode. CSE Report No.132. University of California.

Sachse, P. (1984). Writing assessment in Texas: Practices and problems. Educational Measurement: Issues and Practice, vol. 3, pp. 21-23.

Schaefer, E. (2008). Rater bias patterns in an EFL writing assessment. Language Testing, vol. 25, pp. 465-493.

Stifler, B. (2002). Rhetorical modes.

Tedick, D., J. (1990). ESL writing assessment: Subject-matter knowledge and its impact on performance. English for Specific Purposes, vol. 9, pp. 123-143.

Veal, L. R. & Tillman, M. (1971). Mode of discourse variation in evaluation of childrens writing. Research in the Teaching of English, vol. 5, (1), pp.37-45.

Weigle, S., C. (1999). Investigating rater/prompt interactions in writing assessment: Quantitative and qualitative approaches. Assessing Writing, vol. 6, (2), pp. 145-178.

Weigle, S., C. (2002). Assessing writing. Cambridge University Press.

Weir, C., J. (1990). Communicative language testing. UK: Prentice Hall.

White, E., M. (1982). Some issues in the testing of writing. Notes from the National Testing. Network in Writing. New York: Instructional Resource Center of CUNY.

Yunick, S. (1997). Genres, registers and sociolinguistics. World Englishes, vol.16, (3), pp. 321-336.

Zinsser, W., K. (1988). Writing to learn. New York: Harper & Row.




DOI: https://doi.org/10.24071/llt.v24i2.2986

Article Metrics

Abstract view : 2021 times
PDF view: 472 times

Refbacks

  • There are currently no refbacks.


Copyright (c) 2021 Rania Zribi, Chokri Smaoui

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Indexed and abstracted in:

    

 

LLT Journal Sinta 2 Certificate (S2 = Level 2)

We would like to inform you that LLT Journal: A Journal on Language and Language Teaching has been nationally accredited Sinta 2 by the Ministry of Education, Culture, Research and Technology of the Republic of Indonesia based on the decree  No. Surat Keputusan 158/E/KPT/2021. Validity for 5 years: Vol 23 No 1, 2020 till Vol 27 No 2, 2024

  

 

This work is licensed under CC BY-SA.

Creative Commons Attribution-ShareAlike 4.0 International License

 

Free counters!


 LLT Journal: A Journal on Language and Language Teaching, DOI: https://doi.org/10.24071/llt, e-ISSN 2579-9533 and p-ISSN 1410-7201, is published twice a year, namely in April and October by the English Language Education Study Programme of Teacher Training and Education Faculty of Sanata Dharma University, Yogyakarta, Indonesia.