Assessing inconsistencies in standard setting with the Angoff or Nedelsky technique

Research output: Book/ReportReportOther research output

27 Downloads (Pure)

Abstract

A latent trait method is presented to investigate the possibility that Angoff or Nedelsky judges specify inconsistent probabilities in standard setting techniques for objectives-based instructional programs. It is suggested that judges frequently specify a low probability of success for an easy item but a large probability for a hard item. The responses of 156 pupils to a 25-item test from a tenth grade physics course were inspected by eight Angoff and nine Nedelsky judges. The latent trait analysis produced 18 items showing a satisfactory fit to the Rasch model. Serious errors of specification were found and errors were considerably larger for the Nedelsky technique. Special difficulties with the Nedelsky judges are discussed. Applications of the latent trait method are discussed.
Original languageUndefined
Place of PublicationEnschede, the Netherlands
PublisherUniversity of Twente, Faculty Educational Science and Technology
Publication statusPublished - Mar 1982

Publication series

Name
PublisherUniversity of Twente, Faculty of Educational Science and Technology

Keywords

  • Standard Setting (Scoring)
  • Test Items
  • Test Reliability
  • Cutting Scores
  • Criterion Referenced Tests
  • Mathematical Models
  • Probability
  • Error of Measurement
  • Secondary Education
  • Latent Trait Theory
  • IR-103613
  • Interrater Reliability

Cite this