A method of item pool design is proposed that uses an optimal blueprint for the item pool calculated from the test specifications. The blueprint is a document that specifies the attributes that the items in the computerized adaptive test (CAT) pool should have. The blueprint can be a starting point for the item writing process, and it can be used to assemble item pools in a system of rotating pools from a master pool. The blueprint is also useful for item pool maintenance. Designing the blueprint begins with analyzing the specifications for the CAT, a step amounting to the formation of a classification table involving categorization of quantitative item attributes. Using this table, an integer programming model for the assembly of the shadow tests in the CAT simulation is constructed. An estimate of the ability distribution of the identified population of examinees is obtained, and the CAT simulation is carried out using the integer programming model for the shadow tests and sampling simulees from the ability distribution. The blueprint is then calculated from the counts of the number of items from the cells in the classification table. The best way to implement the blueprint is in a sequential fashion recalculating the blueprint after a certain portion of the items has actually been written and tested so that their attribute values are known.
|Name||OMD research report|
|Publisher||University of Twente, Faculty of Educational Science and Technology|