Constraining item exposure in computerized adaptive testing with shadow tests

Research output: Book/ReportReportProfessional

67 Downloads (Pure)

Abstract

Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter's (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. However, the method does not require time-consuming simulation studies to set values for control parameters prior to the operational use of the test. Instead, the probabilities of item ineligibility can be set "on the fly" using an adaptive procedure based on the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test (LSAT) showed that application of the method yielded perfect control of the item exposure rates and had negligible impact on the bias and MSE functions of the ability estimator.
Original languageEnglish
Place of PublicationEnschede
PublisherOnderwijskundige Meetmethoden en Data-analyse
Number of pages29
Publication statusPublished - 2002

Publication series

NameOMD Research Report
PublisherUniversity of Twente, Faculty of Educational Science and Technology
No.02-06

Fingerprint

method
test
exposure
simulation
rate
school
decision
parameter

Keywords

  • METIS-209504
  • item exposure control
  • IR-103564
  • Computerized Adaptive Testing
  • item-selection constraints
  • shadow test approach
  • randomized control

Cite this

van der Linden, W. J., & Veldkamp, B. P. (2002). Constraining item exposure in computerized adaptive testing with shadow tests. (OMD Research Report; No. 02-06). Enschede: Onderwijskundige Meetmethoden en Data-analyse.
van der Linden, Willem J. ; Veldkamp, Bernard P. / Constraining item exposure in computerized adaptive testing with shadow tests. Enschede : Onderwijskundige Meetmethoden en Data-analyse, 2002. 29 p. (OMD Research Report; 02-06).
@book{2a8d1a2543fd4d2b8cc347dbefed16b4,
title = "Constraining item exposure in computerized adaptive testing with shadow tests",
abstract = "Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter's (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. However, the method does not require time-consuming simulation studies to set values for control parameters prior to the operational use of the test. Instead, the probabilities of item ineligibility can be set {"}on the fly{"} using an adaptive procedure based on the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test (LSAT) showed that application of the method yielded perfect control of the item exposure rates and had negligible impact on the bias and MSE functions of the ability estimator.",
keywords = "METIS-209504, item exposure control, IR-103564, Computerized Adaptive Testing, item-selection constraints, shadow test approach, randomized control",
author = "{van der Linden}, {Willem J.} and Veldkamp, {Bernard P.}",
year = "2002",
language = "English",
series = "OMD Research Report",
publisher = "Onderwijskundige Meetmethoden en Data-analyse",
number = "02-06",

}

van der Linden, WJ & Veldkamp, BP 2002, Constraining item exposure in computerized adaptive testing with shadow tests. OMD Research Report, no. 02-06, Onderwijskundige Meetmethoden en Data-analyse, Enschede.

Constraining item exposure in computerized adaptive testing with shadow tests. / van der Linden, Willem J.; Veldkamp, Bernard P.

Enschede : Onderwijskundige Meetmethoden en Data-analyse, 2002. 29 p. (OMD Research Report; No. 02-06).

Research output: Book/ReportReportProfessional

TY - BOOK

T1 - Constraining item exposure in computerized adaptive testing with shadow tests

AU - van der Linden, Willem J.

AU - Veldkamp, Bernard P.

PY - 2002

Y1 - 2002

N2 - Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter's (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. However, the method does not require time-consuming simulation studies to set values for control parameters prior to the operational use of the test. Instead, the probabilities of item ineligibility can be set "on the fly" using an adaptive procedure based on the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test (LSAT) showed that application of the method yielded perfect control of the item exposure rates and had negligible impact on the bias and MSE functions of the ability estimator.

AB - Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter's (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. However, the method does not require time-consuming simulation studies to set values for control parameters prior to the operational use of the test. Instead, the probabilities of item ineligibility can be set "on the fly" using an adaptive procedure based on the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test (LSAT) showed that application of the method yielded perfect control of the item exposure rates and had negligible impact on the bias and MSE functions of the ability estimator.

KW - METIS-209504

KW - item exposure control

KW - IR-103564

KW - Computerized Adaptive Testing

KW - item-selection constraints

KW - shadow test approach

KW - randomized control

M3 - Report

T3 - OMD Research Report

BT - Constraining item exposure in computerized adaptive testing with shadow tests

PB - Onderwijskundige Meetmethoden en Data-analyse

CY - Enschede

ER -

van der Linden WJ, Veldkamp BP. Constraining item exposure in computerized adaptive testing with shadow tests. Enschede: Onderwijskundige Meetmethoden en Data-analyse, 2002. 29 p. (OMD Research Report; 02-06).