Assessing writing ability in primary education: on the evaluation of text quality and text complexity

Writing is a complex ability, and measuring writing ability is a notoriously complex task. The assessment of writing ability is complicated by the multi-faceted nature of this productive language ability on one hand, and the difficulty of evaluating writing performances on the other hand. In this dissertation, different methods to assess both the quality and the complexity of novice writers’ texts are evaluated with the aim of improving the validity and reliability of writing assessment in primary education. Each of the three methods discussed in this study assesses a different aspect of the construct of writing ability. An anchored analytical assessment (Chapter 2) concerns the quality of a text, with the specific goal of the text considered. Revision tests (Chapter 3) focus on the writers’ evaluation of text quality within the reviewing component of the writing process. Lastly, automated essay evaluation (Chapter 4) evaluates linguistic complexity features of the written product. The results presented in this dissertation indicate several beneficial effects on both reliability and validity when the abovementioned approaches are applied within a writing assessment. First, the addition of anchor essays to an analytical rating procedure was found to improve the agreement amongst raters, the validity of the writing scores, and the extent to which these text quality scores are generalisable across tasks and raters. Second, the use of a constructed-response format offered a valid method to assess the writers’ ability to revise their texts, so that text quality is improved. Finally, the automated evaluation of text complexity features proved promising as an objective metric to evaluate linguistic features which are indicative of the proficiency of novice writers. Overall, the methods evaluated within this study were found to increase both the construct coverage and the consistency of the writing assessment. Therefore, these methods contribute to a valid and reliable assessment of writing ability in primary education.