Quality control of online calibration in computerized assessment

Research output: Book/ReportReportOther research output

13 Downloads (Pure)

Abstract

In computerized adaptive testing, updating item parameter estimates using adaptive testing data is often called online calibration. This study investigated how to evaluate whether the adaptive testing data used for online calibration sufficiently fit the item response model used. Three approaches were investigated, based on a Lagrange multiplier (LM) statistic, a Wald statistic, and a cumulative sum (CUSUM) statistic. The power of the tests was evaluated with a number of simulation studies. It was found that the tests had moderate to good power to detect shifts in the values of the guessing and difficulty parameters, and all tests were equally sensitive to all shifts in the values of all parameters. The practical conclusion is that all of these statistics can be used very well to detect if something has happened to the item parameters but that it may be difficult to attribute the problems to specific parameters.
Original languageUndefined
Place of PublicationNewton, PA, USA
PublisherLaw School Admission Council
Publication statusPublished - Sep 2003

Publication series

NameLSAC research report series
PublisherLaw School Admission Council
No.97-15

Keywords

  • Simulation
  • Models
  • Foreign Countries
  • Adaptive Testing
  • Online Systems
  • Item Response Theory
  • IR-103754
  • Quality Control
  • Test Items
  • Computer Assisted Testing
  • Power (Statistics)

Cite this