The design, development, and validation of a multimedia-based performance assessment for credentialing confined space guards

Sebastiaan de Klerk*, Bernard P. Veldkamp, Theo J.H.M. Eggen

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

2 Citations (Scopus)
9 Downloads (Pure)

Abstract

In this article, we present a study on the design, development, and validation of a multimedia-based performance assessment (MBPA) for measuring the skills of confined space guards in Dutch vocational education. An MBPA is a computer-based assessment that incorporates multimedia to simulate tasks. It is designed to measure performance-based skills. A confined space guard (CSG) supervises operations that are carried out in a confined space (e.g., a tank or silo). In the Netherlands, individuals who want to become certified CSGs must participate in a one-day training program, and pass both a multiple-choice knowledge test and a performance-based assessment. In the first part of this article, we focus on the design and development of the MBPA, using a specific framework for design and development. In the second part of the article, we present a validation study. We use the argument-based approach to validation to validate the MBPA (Kane in Educational measurement. American Council on Education and Praeger Publishers, Westport, 2006 and J Educ Meas 50(1):1–73, 2013). More specifically, the extended argument-based approach to validation is used (Wools et al. in CADMO 18(1):63–82, 2010 and Stud Educ Eval 48:10–18, 2016). The approach suggests using multiple sources of validity evidence to build a comprehensive validity case for the proposed interpretation of assessment scores (Kane 2006, 2013) and to evaluate the strength of the validity case (Wools et al. 2010, 2016). We demonstrate that MBPA scores can be used for their intended purpose; students’ performance in the MBPA can be used as the basis for making a CSG certification decision.

Original languageEnglish
Pages (from-to)565-596
Number of pages32
JournalBehaviormetrika
Volume45
Issue number2
Early online date28 Aug 2018
DOIs
Publication statusPublished - 1 Oct 2018

Keywords

  • UT-Hybrid-D
  • Development
  • Multimedia-based performance assessment
  • Validation
  • Design

Fingerprint Dive into the research topics of 'The design, development, and validation of a multimedia-based performance assessment for credentialing confined space guards'. Together they form a unique fingerprint.

Cite this