A comparison of two procedures for constrained adaptive test construction

Frédéric Robin, Willem J. van der Linden, Daniel R. Eignor, Manfred Steffen, Martha L. Stocking

Research output: Book/ReportReportProfessional

59 Downloads (Pure)

Abstract

The relatively new shadow test approach (STA) to computerized adaptive testing (CAT) proposed by Wim van der Linden is a potentially attractive alternative to the weighted deviation algorithm (WDA) implemented at ETS. However, it has not been evaluated under testing conditions representative of current ETS testing programs. Of interest was whether STA would, under typical high-stakes on-demand testing situations, produce tests of comparable or better psychometric quality as those produced by the current weighted deviation algorithm. Based on simulated data, we found that the STA performed as well or slightly better than the WDA on two of the three commonly accepted testing objectives: measurement and content. The WDA appeared to perform slightly better than the STA when the issue is security or item exposure control. The paper provides a review of the rationale that led to the specific testing objectives employed, an outline of the test construction steps common to the two procedures investigated, and a description of the specific models and algorithms employed by both procedures. Detailed description of the simulation study conducted and the results obtained from both of the procedures for one of the pools are also provided. Results are summarized and further research needs in particular areas discussed.
Original languageEnglish
Place of PublicationPrinceton, New Jersey
PublisherEducational Testing Service
Publication statusPublished - 2004

Publication series

NameETS research report
PublisherUniversity of Twente, Faculty of Educational Science and Technology
No.04-39

Keywords

  • METIS-219564
  • test security
  • IR-103559
  • Automated test assembly
  • Computerized Adaptive Testing
  • Item Response Theory
  • item exposure

Fingerprint Dive into the research topics of 'A comparison of two procedures for constrained adaptive test construction'. Together they form a unique fingerprint.

Cite this