Comparative judgment: Assessing competence development in the context of humanitarian engineering education

Addressing current complex issues characterised by growing populations lacking
access to fundamental resources, conflicts or climate change requires innovative
socio-technological solutions. Thus, there is a need for a new generation of
engineers equipped with competencies to work in challenging, volatile and complex societal contexts. While much work focuses on how can students develop such competencies, assessing the acquisition of these competencies in higher education is still challenging. To address this, new, objective methods beyond traditional testing focused on knowledge acquisition are required. In this research, we investigated whether comparative judgment has the potential to be a more consistent and fair method for summative competency-based assessment. Comparative judgment was used for the summative assessment of a competence self-reflection essay within the challenge-based learning master´s course Introduction to Humanitarian Engineering at the University of Twente. Elevel assessors evaluated 18 essays, making a total of 245 comparisons, and shared their perceived experiences in an online survey. The findings suggest that while assessors generally view comparative judgment positively, it may be better suited for peer learning and formative assessment rather than for summative assessment in challenge-based learning settings.