Detecting Mislabeled Data Using Supervised Machine Learning Techniques

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    3 Downloads (Pure)

    Abstract

    A lot of data sets, gathered for instance during user experiments, are contaminated with noise. Some noise in the measured features is not much of a problem, it even increases the performance of many Machine Learning (ML) techniques. But for noise in the labels (mislabeled data) the situation is quite different, label noise deteriorates the performance of all ML techniques. The research question addressed in this paper is to what extent can one detect mislabeled data using a committee of supervised Machine Learning models. The committee under consideration consists of a Bayesian model, Random Forest, Logistic classifier, a Neural Network and a Support Vector Machine. This committee is applied to a given data set in several iterations of 5-fold Cross validation. If a data sample is misclassified by all committee members in all iterations (consensus) then it is tagged as mislabeled. This approach was tested on the Iris plant data set, which is artificially contaminated with mislabeled data. For this data set the precision of detecting mislabeled samples is 100% and the recall is approximately 5%. The approach was also tested on the Touch data set, a data set of naturalistic social touch gestures. It is known that this data set contains mislabeled data, but the amount is unknown. For this data set the proposed method achieved a precision of 70% and for almost all other tagged samples the corresponding touch gesture deviated a lot from the prototypical touch gesture. Overall the proposed method shows high potential for detecting mislabeled samples, but the precision on other data sets needs to be investigated.
    Original languageEnglish
    Title of host publicationAugmented Cognition. Neurocognition and Machine Learning
    Subtitle of host publication11th International Conference, AC 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9-14, 2017, Proceedings, Part I
    EditorsDylan D. Schmorrow, Cali M. Fidopiastis
    PublisherSpringer
    Pages571-581
    ISBN (Electronic)978-3-319-58628-1
    ISBN (Print)978-3-319-58627-4
    DOIs
    Publication statusPublished - 2017
    Event11th International Conference on Augmented Cognition 2017 - Vancouver Convention Centre, Vancouver, Canada
    Duration: 9 Jul 201714 Jul 2017
    Conference number: 11
    http://2017.hci.international/ac

    Conference

    Conference11th International Conference on Augmented Cognition 2017
    Abbreviated titleHCI
    CountryCanada
    CityVancouver
    Period9/07/1714/07/17
    Internet address

    Keywords

    • Mislabeled data
    • Supervised Machine Learning

    Fingerprint Dive into the research topics of 'Detecting Mislabeled Data Using Supervised Machine Learning Techniques'. Together they form a unique fingerprint.

  • Cite this

    Poel, M. (2017). Detecting Mislabeled Data Using Supervised Machine Learning Techniques. In D. D. Schmorrow, & C. M. Fidopiastis (Eds.), Augmented Cognition. Neurocognition and Machine Learning: 11th International Conference, AC 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9-14, 2017, Proceedings, Part I (pp. 571-581). Springer. https://doi.org/10.1007/978-3-319-58628-1_43