User-based active learning

Active learning has been proven a reliable strategy to reduce manual efforts in training data labeling. Such strategies incorporate the user as oracle: the classifier selects the most appropriate example and the user provides the label. While this approach is tailored towards the classifier, more intelligent input from the user may be beneficial. For instance, given only one example at a time users are hardly able to determine whether this example is an outlier or not. In this paper we propose user-based visually-supported active learning strategies that allow the user to do both, selecting and labeling examples given a trained classifier. While labeling is straightforward, selection takes place using a interactive visualization of the classifier's a-posteriori output probabilities. By simulating different user selection strategies we show, that user-based active learning outperforms uncertainty based sampling methods and yields a more robust approach on different data sets. The obtained results point towards the potential of combining active learning strategies with results from the field of information visualization.