Testing for aberrant behavior in response time modeling

S. Marianti, Gerardus J.A. Fox, Marianna Avetisyan, Bernard P. Veldkamp

Research output: Book/ReportReportProfessional

26 Citations (Scopus)
45 Downloads (Pure)

Abstract

Many standardized tests are now administered via computer rather than paper-and-pencil format. In a computer-based testing environment, it is possible to record not only the test taker’s response to each question (item), but also the amount of time spent by the test taker in considering and answering each item. Response times (RTs) provide information not only about the test taker’s ability and response behavior but also about item and test characteristics. The current study focuses on the use of RTs to detect aberrant test-taker responses. An example of such aberrance is a correct answer with a short response time on a difficult question. Such aberrance may be displayed when a test taker or test takers have preknowledge of the items. Another example is rapid guessing, wherein the test taker displays unusually short response times for a series of items. When rapid guessing occurs at the end of a timed test, it often indicates that the test taker has run out of time before completing the test. In the current study, a model for detecting various types of aberrant RT patterns is proposed and evaluated. In simulation studies, the model was successful in identifying aberrant response patterns. Further investigations are required to analyze flagged patterns more thoroughly, possibly by applying additional information.
Original languageEnglish
Place of PublicationNewton, PA
PublisherLaw School Admission Council
Number of pages32
Publication statusPublished - 2014

Publication series

NameLSAC Research Report Series
PublisherLaw School Admission Council
No.14-02

Fingerprint

Testing

Keywords

  • IR-94422
  • METIS-309343

Cite this

Marianti, S., Fox, G. J. A., Avetisyan, M., & Veldkamp, B. P. (2014). Testing for aberrant behavior in response time modeling. (LSAC Research Report Series; No. 14-02). Newton, PA: Law School Admission Council.
Marianti, S. ; Fox, Gerardus J.A. ; Avetisyan, Marianna ; Veldkamp, Bernard P. / Testing for aberrant behavior in response time modeling. Newton, PA : Law School Admission Council, 2014. 32 p. (LSAC Research Report Series; 14-02).
@book{4347390ac35f44e1852c91fdea74a5b8,
title = "Testing for aberrant behavior in response time modeling",
abstract = "Many standardized tests are now administered via computer rather than paper-and-pencil format. In a computer-based testing environment, it is possible to record not only the test taker’s response to each question (item), but also the amount of time spent by the test taker in considering and answering each item. Response times (RTs) provide information not only about the test taker’s ability and response behavior but also about item and test characteristics. The current study focuses on the use of RTs to detect aberrant test-taker responses. An example of such aberrance is a correct answer with a short response time on a difficult question. Such aberrance may be displayed when a test taker or test takers have preknowledge of the items. Another example is rapid guessing, wherein the test taker displays unusually short response times for a series of items. When rapid guessing occurs at the end of a timed test, it often indicates that the test taker has run out of time before completing the test. In the current study, a model for detecting various types of aberrant RT patterns is proposed and evaluated. In simulation studies, the model was successful in identifying aberrant response patterns. Further investigations are required to analyze flagged patterns more thoroughly, possibly by applying additional information.",
keywords = "IR-94422, METIS-309343",
author = "S. Marianti and Fox, {Gerardus J.A.} and Marianna Avetisyan and Veldkamp, {Bernard P.}",
year = "2014",
language = "English",
series = "LSAC Research Report Series",
publisher = "Law School Admission Council",
number = "14-02",

}

Marianti, S, Fox, GJA, Avetisyan, M & Veldkamp, BP 2014, Testing for aberrant behavior in response time modeling. LSAC Research Report Series, no. 14-02, Law School Admission Council, Newton, PA.

Testing for aberrant behavior in response time modeling. / Marianti, S.; Fox, Gerardus J.A.; Avetisyan, Marianna; Veldkamp, Bernard P.

Newton, PA : Law School Admission Council, 2014. 32 p. (LSAC Research Report Series; No. 14-02).

Research output: Book/ReportReportProfessional

TY - BOOK

T1 - Testing for aberrant behavior in response time modeling

AU - Marianti, S.

AU - Fox, Gerardus J.A.

AU - Avetisyan, Marianna

AU - Veldkamp, Bernard P.

PY - 2014

Y1 - 2014

N2 - Many standardized tests are now administered via computer rather than paper-and-pencil format. In a computer-based testing environment, it is possible to record not only the test taker’s response to each question (item), but also the amount of time spent by the test taker in considering and answering each item. Response times (RTs) provide information not only about the test taker’s ability and response behavior but also about item and test characteristics. The current study focuses on the use of RTs to detect aberrant test-taker responses. An example of such aberrance is a correct answer with a short response time on a difficult question. Such aberrance may be displayed when a test taker or test takers have preknowledge of the items. Another example is rapid guessing, wherein the test taker displays unusually short response times for a series of items. When rapid guessing occurs at the end of a timed test, it often indicates that the test taker has run out of time before completing the test. In the current study, a model for detecting various types of aberrant RT patterns is proposed and evaluated. In simulation studies, the model was successful in identifying aberrant response patterns. Further investigations are required to analyze flagged patterns more thoroughly, possibly by applying additional information.

AB - Many standardized tests are now administered via computer rather than paper-and-pencil format. In a computer-based testing environment, it is possible to record not only the test taker’s response to each question (item), but also the amount of time spent by the test taker in considering and answering each item. Response times (RTs) provide information not only about the test taker’s ability and response behavior but also about item and test characteristics. The current study focuses on the use of RTs to detect aberrant test-taker responses. An example of such aberrance is a correct answer with a short response time on a difficult question. Such aberrance may be displayed when a test taker or test takers have preknowledge of the items. Another example is rapid guessing, wherein the test taker displays unusually short response times for a series of items. When rapid guessing occurs at the end of a timed test, it often indicates that the test taker has run out of time before completing the test. In the current study, a model for detecting various types of aberrant RT patterns is proposed and evaluated. In simulation studies, the model was successful in identifying aberrant response patterns. Further investigations are required to analyze flagged patterns more thoroughly, possibly by applying additional information.

KW - IR-94422

KW - METIS-309343

M3 - Report

T3 - LSAC Research Report Series

BT - Testing for aberrant behavior in response time modeling

PB - Law School Admission Council

CY - Newton, PA

ER -

Marianti S, Fox GJA, Avetisyan M, Veldkamp BP. Testing for aberrant behavior in response time modeling. Newton, PA: Law School Admission Council, 2014. 32 p. (LSAC Research Report Series; 14-02).