Comparing simulated and theoretical sampling distributions of the U3 person-fit statistic

The accuracy with which the theoretical sampling distribution of van der Flier’s person-fit statistic U3 approaches the empirical U3 sampling distribution is affected by the item discrimination. A simulation study showed that for tests with a moderate or a strong mean item discrimination, the Type I error rates were either too high or too low to be used in practice. It was concluded that the use of standard normal deviates for the standardized version of the U3 statistic may be problematic. Nevertheless, the U3 statistic is suitable for evaluating the relative likelihood of item score patterns, for example, if one wishes to select a fixed percentage of the most improbable patterns.