The impact of parameter estimation on computerized adaptive testing with item cloning

Research output: Book/ReportReportOther research output

Abstract

Item cloning techniques can greatly reduce the cost of item writing and enhance the flexibility of item presentation. An important consequence of cloning is that it may cause variability in the item parameters. Recently, Glas and van der Linden (in press, 2005) proposed a multilevel item response model where it is assumed that the item parameters of a 3-parameter logistic (3PL) model or a 3-parameter normal ogive (3PNO) model are sampled from a multivariate normal distribution associated with a parent item. In the sequel, the model will be referred to as the item cloning model, which will be abbreviated ICM. Several procedures for item bank calibration and computerized adaptive testing (CAT) were proposed. The latter procedures were developed under the usual assumption that the item parameters are known. However, in practice, item parameters have to be estimated, which introduces an error component that can have substantial effects. For the standard 3PL model, van der Linden and Glas (2000, 2001) show that capitalization on estimation error can lead to a substantial loss of precision. In the present report, this finding is corroborated for the ICM. It is shown that the problem can be solved by a Bayesian item selection procedure where the uncertainty about the item parameters is taken into account by implicating their posterior distributions. These posterior distributions are generated using the Gibbs Sampler. A simulation study is presented to illustrate the performance of the method.
Original languageUndefined
Place of PublicationNewton, PA, USA
PublisherLaw School Admission Council
Number of pages7
Publication statusPublished - Nov 2005

Publication series

NameLSAC research report series
PublisherLaw School Admission Council
No.02-06

Keywords

  • IR-104257

Cite this

Glas, C. A. W. (2005). The impact of parameter estimation on computerized adaptive testing with item cloning. (LSAC research report series; No. 02-06). Newton, PA, USA: Law School Admission Council.
Glas, Cornelis A.W. / The impact of parameter estimation on computerized adaptive testing with item cloning. Newton, PA, USA : Law School Admission Council, 2005. 7 p. (LSAC research report series; 02-06).
@book{2ae862a8ceea46e4a98bef35cfbcd3db,
title = "The impact of parameter estimation on computerized adaptive testing with item cloning",
abstract = "Item cloning techniques can greatly reduce the cost of item writing and enhance the flexibility of item presentation. An important consequence of cloning is that it may cause variability in the item parameters. Recently, Glas and van der Linden (in press, 2005) proposed a multilevel item response model where it is assumed that the item parameters of a 3-parameter logistic (3PL) model or a 3-parameter normal ogive (3PNO) model are sampled from a multivariate normal distribution associated with a parent item. In the sequel, the model will be referred to as the item cloning model, which will be abbreviated ICM. Several procedures for item bank calibration and computerized adaptive testing (CAT) were proposed. The latter procedures were developed under the usual assumption that the item parameters are known. However, in practice, item parameters have to be estimated, which introduces an error component that can have substantial effects. For the standard 3PL model, van der Linden and Glas (2000, 2001) show that capitalization on estimation error can lead to a substantial loss of precision. In the present report, this finding is corroborated for the ICM. It is shown that the problem can be solved by a Bayesian item selection procedure where the uncertainty about the item parameters is taken into account by implicating their posterior distributions. These posterior distributions are generated using the Gibbs Sampler. A simulation study is presented to illustrate the performance of the method.",
keywords = "IR-104257",
author = "Glas, {Cornelis A.W.}",
year = "2005",
month = "11",
language = "Undefined",
series = "LSAC research report series",
publisher = "Law School Admission Council",
number = "02-06",

}

Glas, CAW 2005, The impact of parameter estimation on computerized adaptive testing with item cloning. LSAC research report series, no. 02-06, Law School Admission Council, Newton, PA, USA.

The impact of parameter estimation on computerized adaptive testing with item cloning. / Glas, Cornelis A.W.

Newton, PA, USA : Law School Admission Council, 2005. 7 p. (LSAC research report series; No. 02-06).

Research output: Book/ReportReportOther research output

TY - BOOK

T1 - The impact of parameter estimation on computerized adaptive testing with item cloning

AU - Glas, Cornelis A.W.

PY - 2005/11

Y1 - 2005/11

N2 - Item cloning techniques can greatly reduce the cost of item writing and enhance the flexibility of item presentation. An important consequence of cloning is that it may cause variability in the item parameters. Recently, Glas and van der Linden (in press, 2005) proposed a multilevel item response model where it is assumed that the item parameters of a 3-parameter logistic (3PL) model or a 3-parameter normal ogive (3PNO) model are sampled from a multivariate normal distribution associated with a parent item. In the sequel, the model will be referred to as the item cloning model, which will be abbreviated ICM. Several procedures for item bank calibration and computerized adaptive testing (CAT) were proposed. The latter procedures were developed under the usual assumption that the item parameters are known. However, in practice, item parameters have to be estimated, which introduces an error component that can have substantial effects. For the standard 3PL model, van der Linden and Glas (2000, 2001) show that capitalization on estimation error can lead to a substantial loss of precision. In the present report, this finding is corroborated for the ICM. It is shown that the problem can be solved by a Bayesian item selection procedure where the uncertainty about the item parameters is taken into account by implicating their posterior distributions. These posterior distributions are generated using the Gibbs Sampler. A simulation study is presented to illustrate the performance of the method.

AB - Item cloning techniques can greatly reduce the cost of item writing and enhance the flexibility of item presentation. An important consequence of cloning is that it may cause variability in the item parameters. Recently, Glas and van der Linden (in press, 2005) proposed a multilevel item response model where it is assumed that the item parameters of a 3-parameter logistic (3PL) model or a 3-parameter normal ogive (3PNO) model are sampled from a multivariate normal distribution associated with a parent item. In the sequel, the model will be referred to as the item cloning model, which will be abbreviated ICM. Several procedures for item bank calibration and computerized adaptive testing (CAT) were proposed. The latter procedures were developed under the usual assumption that the item parameters are known. However, in practice, item parameters have to be estimated, which introduces an error component that can have substantial effects. For the standard 3PL model, van der Linden and Glas (2000, 2001) show that capitalization on estimation error can lead to a substantial loss of precision. In the present report, this finding is corroborated for the ICM. It is shown that the problem can be solved by a Bayesian item selection procedure where the uncertainty about the item parameters is taken into account by implicating their posterior distributions. These posterior distributions are generated using the Gibbs Sampler. A simulation study is presented to illustrate the performance of the method.

KW - IR-104257

M3 - Report

T3 - LSAC research report series

BT - The impact of parameter estimation on computerized adaptive testing with item cloning

PB - Law School Admission Council

CY - Newton, PA, USA

ER -

Glas CAW. The impact of parameter estimation on computerized adaptive testing with item cloning. Newton, PA, USA: Law School Admission Council, 2005. 7 p. (LSAC research report series; 02-06).