TY - GEN
T1 - ASSESSMENT OF COMPETENCY DEVELOPMENT IN A CHALLENGE BASED LEARNING COURSE
T2 - CAN COACHES BE OBJECTIVE ASSESSORS?
AU - Petrova, N.
AU - Chapel, Leonie
AU - Buunk, Luuk
AU - Kaptijn, G.H.
PY - 2022
Y1 - 2022
N2 - Higher education institutions aim to incorporate competency development into theirengineering curricula, which can help engineering students become independentcritical thinkers with entrepreneurial mindsets. However, no solid methods exist toevaluate the acquisition of these competencies. Such assessments’ objectivities areoften ensured by distinguishing between who supervises a student group and whogrades its project. The assessor’s active involvement in the learning process isessential for assessing competency development during the learning process, but suchinvolvement may lead to assessor bias. This study aims to investigate whether andunder what conditions coaches can be objective assessors. An intraclass correlationcoefficient (ICC) was used to measure the level of agreement between assessors andcoaches when using the same rubric to assess students’ deliverables. Four assessorsand seven coaches from the University of Twente assessed 24 students’ individuallearning processes based on individual reflection deliverables. The coaches assessedthe students they supervised during a challenge-based learning (CBL) course, while the assessors were without participating in the learning process assigned randomly tostudents. The means were compared using SPSS, which indicated, among otherthings, that coaches generally awarded higher scores than assessors. This mayindicate that coaches are biased because of their involvement in the learning process.Despite this, the results also indicate that coach assessment was in line with assessorswhen the coach was an appointed and experienced examiner.
AB - Higher education institutions aim to incorporate competency development into theirengineering curricula, which can help engineering students become independentcritical thinkers with entrepreneurial mindsets. However, no solid methods exist toevaluate the acquisition of these competencies. Such assessments’ objectivities areoften ensured by distinguishing between who supervises a student group and whogrades its project. The assessor’s active involvement in the learning process isessential for assessing competency development during the learning process, but suchinvolvement may lead to assessor bias. This study aims to investigate whether andunder what conditions coaches can be objective assessors. An intraclass correlationcoefficient (ICC) was used to measure the level of agreement between assessors andcoaches when using the same rubric to assess students’ deliverables. Four assessorsand seven coaches from the University of Twente assessed 24 students’ individuallearning processes based on individual reflection deliverables. The coaches assessedthe students they supervised during a challenge-based learning (CBL) course, while the assessors were without participating in the learning process assigned randomly tostudents. The means were compared using SPSS, which indicated, among otherthings, that coaches generally awarded higher scores than assessors. This mayindicate that coaches are biased because of their involvement in the learning process.Despite this, the results also indicate that coach assessment was in line with assessorswhen the coach was an appointed and experienced examiner.
KW - Challenge based learning
KW - Assessment of competences
KW - Learning impact
KW - objective assessment
KW - Learning process
KW - Lifelong learning skills
M3 - Conference contribution
BT - Towards a New Future in Engineering Education
PB - Societe Europeenne pour la Formation des Ingenieurs (SEFI)
CY - Barcelona
ER -