Summative Assessment, Test Scores and Text Quality: A Study of Cohesion as an Unspecified Descriptor in the Assessment Scale
Summative assessment of students' writing predicts not only the extent to which the course learning objectives have been achieved but also reveals.
- Pub. date: April 15, 2020
- Pages: 523-535
- 640 Downloads
- 836 Views
- 3 Citations
Summative assessment of students' writing predicts not only the extent to which the course learning objectives have been achieved but also reveals the relevance of the assessment design with the construct of writing being assessed. Any dichotomy between the assessment criteria and the construct of writing or between the assessment criteria and test scoring procedures can produce unreliable and invalid interpretations of the students' writing proficiency. Assuming cohesion as a measure of writing quality, the present study chose samples of academic writing which did not specify cohesion as a descriptor in the assessment scale. A cohesion index was, therefore, developed to investigate how cohesive devices created texture in the sample texts and correlated with the test scores. Descriptive statistics and non-parametric correlation analysis revealed that cohesive devices were positively associated with appropriate use to create texture; however, they only weakly correlated with the test scores. The findings imply that there is the need for developing assessment criteria which consistently measures the text-forming resources to reliably ascertain the writing proficiency of the students. The study recommends a research initiative based on an analytical assessment criteria to ensure a more accurate analysis of the role of cohesion in text-formation and writing quality.
Keywords: Academic writing, cohesion, English as a Foreign Language, language testing, summative assessment, texture, writing quality
References
Ahmad, Z. (2020). Peer observation as a professional development intervention in EFL pedagogy: A case of a reading lesson on developing the top-down processing skills of the preparatory year students. International Linguistics Research, 3(1), 1-15. https://doi.org/10.30560/ilr.v3n1p1
Ahmad, Z. (2019). Teacher beliefs about students’ use of cohesion in writing: What does the textual evidence reveal? Journal of Language and Education, 5(4), 9-24.https://doi.org/10.17323/jle.2019.9708
Ahmad, Z. (2019). Analyzing argumentative essay as an academic genre on assessment frameworks of IELTS and TOEFL. In S. Hidri (Ed.), English language teaching research in the Middle East and North Africa: Multiple perspectives (pp. 279-299). Palgrave Macmillan. https://doi.org/10.1007/978-3-319-98533-6
Al-Khatib, H. (2017). The five tier model for teaching English academic writing in EFL contexts. Arab World English Journal, 8(2), 74-86. https://doi.org/10.24093/awej/vol8no2.5
Aydin, U. (2019). Test anxiety: Gender differences in elementary school students. European Journal of Educational Research, 8(1), 21-30. https://doi.org/10.12973/eu-jer.8.1.21
Bachman, L. F., &Palmer, A.S. (1996).Language testing in practice. Oxford University Press.
Becker, A. (2011). Examining rubrics used to measure writing performance in US intensive English programs. The CATESOL Journal, 22(1), 113–130.
Benzehaf, B. (2017). Exploring teachers’ assessment practices and skills. International Journal of Assessment Tools in Education, 4(1), 1-18. https://doi.org/10.21449/ijate.254581
Best, J. W., & Kahn, J. V. (2003).Research in education (9th ed.). Allyn and Bacon.
Black, P., & Wiliam, D. (2018).Classroom assessment and pedagogy. Assessment in Education: Principles, Policy & Practice, 25(6), 551-575. https://doi.org/10.1080/0969594X.2018.1441807
Breland, H. M., Kubota, M. Y., & Bonner, M. W. (1999).The performance assessment study in writing: Analysis of the SAT II: Writing subject test (College Board Report No. 99-4). College Entrance Examination Board.
Cameron, C. A., Lee, K., Webster, S., Munro, K., Hunt, A. K., & Linton, M. J. (1995). Text cohesion in children’s narrative writing. Applied Psycholinguistics, 16, 257-269.
Chiang, S. Y. (1999). Assessing grammatical and textual features in L2 writing samples: The case of French as a foreign language. The Modern Language Journal, 83(2), 219-232.
Chiang, S. Y. (2003). The importance of cohesive conditions to perceptions of writing quality at the early stages of foreign language learning. System, 31(4), 471-484.
Chodorow, M., & Burnstein, J. (2004).Beyond essay length: Evaluating e-rater’s performance on TOEFL essays (ETS Research Report No.RR-04-04). ETS.
Cox, B. E., Shanahan, T., & Sulzby, E. (1990). Good and poor elementary readers' use of cohesion in writing. Reading Research Quarterly, 25, 47-65.
Cox, B. E., Shanahan, T., & Tinzmann, M. B. (1991).Children's knowledge of organization, cohesion, and voice in written exposition. Research in the teaching of English, 25(2), 179-218.
Crusan, D., Plakans, L., & Gebril, A. (2016).Writing assessment literacy: Surveying second language teachers’ knowledge, beliefs, and practices. Assessing Writing, 28, 43-56. https://doi.org/10.1016/j.asw.2016.03.001
Darweesh, A. D., & Kadhimi, S. A. H. (2016). Iraqi EFL learners' problems in using conjunctions as cohesive devices. Journal of Education and Practice, 7(11), 169-180.
deHaan, P., & van Esch, K. (2008). Measuring and assessing the development of foreign language writing competence. Porta Linguiarium, 9, 7-21.
Demir, S., & Erdogan, A. (2018). The role of teaching grammar in first language education. European Journal of Educational Research, 7(1), 87-101. https://doi.org/10.12973/eu-jer.7.1.87
Dornyei, Z. (2007). Research methods in applied linguistics. Oxford University Press.
Gay, L.R. (2005). Educational research (5th ed.). National Book Foundation.
Gilquin, G., Granger, S., & Paquot, M. (2007). Learner corpora: The missing link in EAP pedagogy. Journal of English for Academic Purposes, 6, 319-335.
Guiju, Z. (2005). The cohesive knowledge and English writing quality of college student. CELEA Journal, 28(3), 24-30.
Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. Longman.
Halliday, M. A. K., & Mathiessen, C. (2004). An introduction to functional grammar. Hodder Arnold.
Hamp-Lyons, L. (2003). Writing teachers as assessors of writing. In B. Kroll (Ed.) Exploring the dynamics of second language writing (pp.162-189).Cambridge University Press.
Hanusz, Z., & Tarasinska, J. (2015). Normalization of the Kolmogorov–Smirnov and Shapiro–Wilk tests of normality. Biometrical Letters, 52(2), 85-93. https://doi.org/10.1515/bile-2015-0008
Hinkel, E. (2001). Matters of cohesion in L2 academic texts. Applied Language Learning, 12(2), 111-132.
Hoey, M. (1991a). Patterns of lexis in text. Oxford University Press.
Hughes, A. (2002). Testing for language teachers (2nd ed.). Cambridge University Press.
Huot, B. (2002). (Re) articulating writing assessment for teaching and learning. Logan.
Hyland, K. (2006). English for academic purposes: an advanced resource book. Routledge.
Iliya, A. (2014).Formative and summative assessment in educational enterprise, Journal of Education and Practice, 5(20), 111-117.
Jafarpur, A. (1991). Cohesiveness as a basis for evaluating compositions. System, 19(4), 459-465.
Johnson, P. (1992). Cohesion and coherence in compositions in Malay and English. RELC Journal, 23(2), 1-17.
Kalajahi, S. A. R., & Abdullah, A.N. (2016). Assessing assessment literacy and practices among lecturers, Pedagogy, 124(4), 232–248.
Lee, I. (1998). Writing in the Hong Kong secondary classroom: Teachers' beliefs and practice. Hong Journal of Applied Linguistics, 3, 61-76.
Lee, I. (2010). Writing teacher education and teacher learning: Testimonies of four EFL teachers. Journal of Second Language Writing, 19(3), 143–157.
Lincoln, Y. S., & Gaba, E. G. (1985). Naturalistic inquiry. Sage Publications.
Liu, M., & Braine G. (2005). Cohesive features in argumentative writing produced by Chinese undergraduates. System, 33, 623-636.
Llach, M. P. A., & Catalan, R. M. J. (2007). Lexical reiteration in EFL young learners’ essays: Does it relate to the type of instruction? International Journal of English Studies, 7(2), 85-103. https://doi.org/10.6018/ijes.7.2.49001
Lynne, P. (2004). Coming to terms: A theory of writing assessment. Utah State University Press.
Martin, J. R. (2001). Cohesion and texture. In D. Schiffrin, D. Tannen, & H. Hamilton (Eds.), The handbook of discourse analysis (pp. 35-53). Blackwell.
McCarthy, M. (1991). Discourse Analysis for Language Teachers. Cambridge University Press.
McCutchen, D., & Perfetti, C. A. (1982). Coherence and connectedness in the development of discourse production. Text-Interdisciplinary Journal for the Study of Discourse, 2(1–3), 113–140.
Mohamed, A. H., & Omer, M. R. (2000). Texture and culture: Cohesion as a marker in rhetorical organization in Arabic and English narrative texts. RELC Journal, 31(2), 45-75.
Mohamed, N. (2016). Use of conjunctions in argumentative essay by ESL undergraduates. e-Academia Journal UiTMT, 5(1), 1-13.
Oshima, A., & Hogue, A. (2006).Writing academic English (4th ed.). Longman.
Rahman, Z. A. A. (2013). The use of cohesive devices writing by Omani student-teachers. SAGE Open, 3, 1-10.
Sekaran, U. (2006). Research methods for business: A skill building approach. John Wiley & Sons.
Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3/4), 591-611.
Song, M., & Xia, W. (2002). Combination of textual cohesive ties and textual teaching for the teaching of English writing: a statistical analysis of the good and poor compositions written by non-English major freshmen. Foreign Language World, 6, 40-44.
States, J., Detrich, R., & Keyworth, R. (2018). Overview of summative assessment. The Wing Institute.
Stiggins, R. (2001). Student involved classroom assessment. Merrill Publishing.
Sultana, N. (2019). Language assessment literacy: An uncharted area for the English language teachers in Bangladesh, Language Testing in Asia, 9(1), 2-14. https://doi.org/10.1186/s40468-019-0077-8
Tanskanen, S. K. (2006). Collaborating towards coherence. John Benjamins. https://doi.org/10.1075/pbns.146
Ting, F. (2003). An Investigation of cohesive errors in the writing of PRC tertiary EFL students (Unpublished master’s thesis). National University of Singapore.
Todd, W., Khongput, R., & Darasawang, P. (2007).Coherence, cohesion and comments on students' academic essays. Assessing Writing, 12(1), 10-25. https://doi.org/10.1016/j.asw.2007.02.002
Wahby, M. (2014). The effect of implementing cohesive ties by Saudi prep-year pre intermediate students on their written texts. European Scientific Journal, 10(4), 220-232.
Weigle, S. C. (2007). Teaching writing teachers about assessment. Journal of Second Language Writing, 16(3), 194–209. https://doi.org/10.1016/j.jslw.2007.07.004
White, E. (2009). Are you assessment literate? Some fundamental questions regarding effective classroom-based assessment. OnCUE Journal, 3(1), 3–25.
Witte, S., & Faigley, L. (1981). Coherence, cohesion, and writing ability. College Composition and Communication, 32, 189-204.
Zhang, M. (2000). Cohesive features in exploratory writing of undergraduates in two Chinese universities. RELC Journal, 31, 61–93.