All about validity: an evaluation system for the quality of educational assessment

S. Wools

Research output: ThesisPhD Thesis - Research UT, graduation UTAcademic

86 Downloads (Pure)

Abstract

At all levels of education, tests and assessments are used to gather information about students’ skills and competences. This information can be used for decisions about groups of students or about individual students. When the stakes of these decisions are high, it becomes increasingly important that the assessments used are of good quality. In this dissertation, assessment quality is defined as the degree to which an assessment instrument is appropriate for its intended purpose. This definition entails a flexible view of quality: dependent on the intended interpretation and use of test scores, different quality criteria should be used for evaluation. This approach is also described as an argument-based approach to quality. It originates in validity theory, and our research project aimed to extend it to quality. This is done in a design-based research project whose aim was to develop an evaluation system, which incorporates an argument-based approach to quality, for the quality of educational assessments. In this design-based research project, a theoretical foundation is developed. This theoretical foundation describes the argument-based approach to quality for a single test. It also describes this approach in a context of an assessment program where multiple assessment instances are combined into one decision. From this theoretical foundation, design principles are formulated and then used to build a prototype of the evaluation system. This prototype is then evaluated and adjusted in an iterative process. This dissertation describes the theoretical foundation, the design principles, and several iterations in the development of the prototype.
Original languageEnglish
Awarding Institution
  • University of Twente
Supervisors/Advisors
  • Eggen, Theo T.J.H.M., Supervisor
Award date26 Jun 2015
Place of PublicationEnschede
Publisher
Print ISBNs978-94-6259-709-9
Publication statusPublished - 26 Jun 2015

Fingerprint

evaluation
research project
earning a doctorate
group decision
student
level of education
interpretation

Keywords

  • METIS-310792
  • IR-96184

Cite this

Wools, S.. / All about validity: an evaluation system for the quality of educational assessment. Enschede : Universiteit Twente, 2015. 221 p.
@phdthesis{cbfe2d0f26644fc194c8780e5b51e97d,
title = "All about validity: an evaluation system for the quality of educational assessment",
abstract = "At all levels of education, tests and assessments are used to gather information about students’ skills and competences. This information can be used for decisions about groups of students or about individual students. When the stakes of these decisions are high, it becomes increasingly important that the assessments used are of good quality. In this dissertation, assessment quality is defined as the degree to which an assessment instrument is appropriate for its intended purpose. This definition entails a flexible view of quality: dependent on the intended interpretation and use of test scores, different quality criteria should be used for evaluation. This approach is also described as an argument-based approach to quality. It originates in validity theory, and our research project aimed to extend it to quality. This is done in a design-based research project whose aim was to develop an evaluation system, which incorporates an argument-based approach to quality, for the quality of educational assessments. In this design-based research project, a theoretical foundation is developed. This theoretical foundation describes the argument-based approach to quality for a single test. It also describes this approach in a context of an assessment program where multiple assessment instances are combined into one decision. From this theoretical foundation, design principles are formulated and then used to build a prototype of the evaluation system. This prototype is then evaluated and adjusted in an iterative process. This dissertation describes the theoretical foundation, the design principles, and several iterations in the development of the prototype.",
keywords = "METIS-310792, IR-96184",
author = "S. Wools",
year = "2015",
month = "6",
day = "26",
language = "English",
isbn = "978-94-6259-709-9",
publisher = "Universiteit Twente",
school = "University of Twente",

}

All about validity: an evaluation system for the quality of educational assessment. / Wools, S.

Enschede : Universiteit Twente, 2015. 221 p.

Research output: ThesisPhD Thesis - Research UT, graduation UTAcademic

TY - THES

T1 - All about validity: an evaluation system for the quality of educational assessment

AU - Wools, S.

PY - 2015/6/26

Y1 - 2015/6/26

N2 - At all levels of education, tests and assessments are used to gather information about students’ skills and competences. This information can be used for decisions about groups of students or about individual students. When the stakes of these decisions are high, it becomes increasingly important that the assessments used are of good quality. In this dissertation, assessment quality is defined as the degree to which an assessment instrument is appropriate for its intended purpose. This definition entails a flexible view of quality: dependent on the intended interpretation and use of test scores, different quality criteria should be used for evaluation. This approach is also described as an argument-based approach to quality. It originates in validity theory, and our research project aimed to extend it to quality. This is done in a design-based research project whose aim was to develop an evaluation system, which incorporates an argument-based approach to quality, for the quality of educational assessments. In this design-based research project, a theoretical foundation is developed. This theoretical foundation describes the argument-based approach to quality for a single test. It also describes this approach in a context of an assessment program where multiple assessment instances are combined into one decision. From this theoretical foundation, design principles are formulated and then used to build a prototype of the evaluation system. This prototype is then evaluated and adjusted in an iterative process. This dissertation describes the theoretical foundation, the design principles, and several iterations in the development of the prototype.

AB - At all levels of education, tests and assessments are used to gather information about students’ skills and competences. This information can be used for decisions about groups of students or about individual students. When the stakes of these decisions are high, it becomes increasingly important that the assessments used are of good quality. In this dissertation, assessment quality is defined as the degree to which an assessment instrument is appropriate for its intended purpose. This definition entails a flexible view of quality: dependent on the intended interpretation and use of test scores, different quality criteria should be used for evaluation. This approach is also described as an argument-based approach to quality. It originates in validity theory, and our research project aimed to extend it to quality. This is done in a design-based research project whose aim was to develop an evaluation system, which incorporates an argument-based approach to quality, for the quality of educational assessments. In this design-based research project, a theoretical foundation is developed. This theoretical foundation describes the argument-based approach to quality for a single test. It also describes this approach in a context of an assessment program where multiple assessment instances are combined into one decision. From this theoretical foundation, design principles are formulated and then used to build a prototype of the evaluation system. This prototype is then evaluated and adjusted in an iterative process. This dissertation describes the theoretical foundation, the design principles, and several iterations in the development of the prototype.

KW - METIS-310792

KW - IR-96184

M3 - PhD Thesis - Research UT, graduation UT

SN - 978-94-6259-709-9

PB - Universiteit Twente

CY - Enschede

ER -