Outlier detection in high-stakes college entrance testing

R.R. Meijer

Research output: Book/ReportReportOther research output

Abstract

In this study we discuss recent developments of person-fit analysis in the context of computerized adaptive testing (CAT). Methods from statistical process control are discussed that have been proposed to classify an item score pattern as fitting or misfitting the underlying item response theory (IRT) model in a CAT. Most person-fit research in CAT is restricted to simulated data. In this study, empirical data from a high-stakes test are used. Alternative methods to generate norm distributions to allow the determination of bounds are discussed. These bounds may be used to classify item score patterns as fitting or misfitting. Using bounds determined from the sample, the empirical analysis indicated that different types of misfit can be distinguished. Possibilities to use this method as a diagnostic instrument are discussed.
Original languageUndefined
Place of PublicationNewton, PA, USA
PublisherLaw School Admission Council
Number of pages11
Publication statusPublished - Sep 2005

Publication series

NameLSAC research report series
PublisherLaw School Admission Council
No.01-08

Keywords

  • IR-104281

Cite this

Meijer, R. R. (2005). Outlier detection in high-stakes college entrance testing. (LSAC research report series; No. 01-08). Newton, PA, USA: Law School Admission Council.
Meijer, R.R. / Outlier detection in high-stakes college entrance testing. Newton, PA, USA : Law School Admission Council, 2005. 11 p. (LSAC research report series; 01-08).
@book{edc15cd3345242499524159120af4724,
title = "Outlier detection in high-stakes college entrance testing",
abstract = "In this study we discuss recent developments of person-fit analysis in the context of computerized adaptive testing (CAT). Methods from statistical process control are discussed that have been proposed to classify an item score pattern as fitting or misfitting the underlying item response theory (IRT) model in a CAT. Most person-fit research in CAT is restricted to simulated data. In this study, empirical data from a high-stakes test are used. Alternative methods to generate norm distributions to allow the determination of bounds are discussed. These bounds may be used to classify item score patterns as fitting or misfitting. Using bounds determined from the sample, the empirical analysis indicated that different types of misfit can be distinguished. Possibilities to use this method as a diagnostic instrument are discussed.",
keywords = "IR-104281",
author = "R.R. Meijer",
year = "2005",
month = "9",
language = "Undefined",
series = "LSAC research report series",
publisher = "Law School Admission Council",
number = "01-08",

}

Meijer, RR 2005, Outlier detection in high-stakes college entrance testing. LSAC research report series, no. 01-08, Law School Admission Council, Newton, PA, USA.

Outlier detection in high-stakes college entrance testing. / Meijer, R.R.

Newton, PA, USA : Law School Admission Council, 2005. 11 p. (LSAC research report series; No. 01-08).

Research output: Book/ReportReportOther research output

TY - BOOK

T1 - Outlier detection in high-stakes college entrance testing

AU - Meijer, R.R.

PY - 2005/9

Y1 - 2005/9

N2 - In this study we discuss recent developments of person-fit analysis in the context of computerized adaptive testing (CAT). Methods from statistical process control are discussed that have been proposed to classify an item score pattern as fitting or misfitting the underlying item response theory (IRT) model in a CAT. Most person-fit research in CAT is restricted to simulated data. In this study, empirical data from a high-stakes test are used. Alternative methods to generate norm distributions to allow the determination of bounds are discussed. These bounds may be used to classify item score patterns as fitting or misfitting. Using bounds determined from the sample, the empirical analysis indicated that different types of misfit can be distinguished. Possibilities to use this method as a diagnostic instrument are discussed.

AB - In this study we discuss recent developments of person-fit analysis in the context of computerized adaptive testing (CAT). Methods from statistical process control are discussed that have been proposed to classify an item score pattern as fitting or misfitting the underlying item response theory (IRT) model in a CAT. Most person-fit research in CAT is restricted to simulated data. In this study, empirical data from a high-stakes test are used. Alternative methods to generate norm distributions to allow the determination of bounds are discussed. These bounds may be used to classify item score patterns as fitting or misfitting. Using bounds determined from the sample, the empirical analysis indicated that different types of misfit can be distinguished. Possibilities to use this method as a diagnostic instrument are discussed.

KW - IR-104281

M3 - Report

T3 - LSAC research report series

BT - Outlier detection in high-stakes college entrance testing

PB - Law School Admission Council

CY - Newton, PA, USA

ER -

Meijer RR. Outlier detection in high-stakes college entrance testing. Newton, PA, USA: Law School Admission Council, 2005. 11 p. (LSAC research report series; 01-08).