Applications of decision theory to test-based decision making

The use of Bayesian decision theory to solve problems in test-based decision making is discussed. Four basic decision problems are distinguished: (1) selection; (2) mastery; (3) placement; and (4) classification, the situation where each treatment has its own criterion. Each type of decision can be identified as a specific configuration of one or more of the following elements: a test that provides the scores on which the decisions are based; one or more treatments with respect to which decisions are made; and one or more criteria by which the successes of treatments are measured. For each type of decision, further restrictions or generalizations may hold, such as multivariate test scores, sequential testing, multiple criteria, multiple populations, and quota restrictions. In some applications, combinations of the basic types of decisions may occur. Samples of decision problems illustrate the optimization of the Bayes utility for each possible decision. Examples are given for selection decisions with linear utility, mastery decisions with threshold utility, placement decisions with normal-ogive utility, classification decisions with threshold utility, and combinations of basic decisions. Nine figures illustrate decision systems, and one table gives data for an application.