Machines outperform lay persons in recognizing emotions elicited by autobiographical recollection

J.H. Janssen, P. Tacken, G.J. de Vries, Egon van den Broek, J.H.D.M. Westerink, P. Haselager, W.A. IJsselsteijn

    Research output: Contribution to journalArticleAcademicpeer-review

    26 Citations (Scopus)

    Abstract

    Over the last decade, an increasing number of studies have focused on automated recognition of human emotions by machines. However, performances of machine emotion recognition studies are difficult to interpret because benchmarks have not been established. In order to provide such a benchmark, we compared machine with human emotion recognition. We gathered facial expressions, speech, and physiological signals from 17 individuals expressing 5 different emotional states. Support vector machines achieved an 82% recognition accuracy based on a physiological and facial features. In experiments with 75 humans on the same data, a maximum recognition accuracy of 62.8% was obtained. As machines outperformed humans, automated emotion recognition might be ready to be tested in more practical applications.
    Original languageUndefined
    Pages (from-to)479-517
    Number of pages39
    JournalHuman-computer interaction
    Volume28
    Issue number6
    DOIs
    Publication statusPublished - 29 Jul 2013

    Keywords

    • EWI-21692
    • IR-87031
    • HMI-CI: Computational Intelligence
    • HMI-SLT: Speech and Language Technology
    • HMI-MI: MULTIMODAL INTERACTIONS
    • Vision
    • Speech
    • Physiology
    • machine
    • autobiographical recollection
    • (Human)
    • BioSignals
    • Affect
    • Machine Learning
    • Audio
    • Multimodal
    • Facial expressions
    • Emotion
    • METIS-297587
    • HMI-HF: Human Factors

    Cite this

    Janssen, J. H., Tacken, P., de Vries, G. J., van den Broek, E., Westerink, J. H. D. M., Haselager, P., & IJsselsteijn, W. A. (2013). Machines outperform lay persons in recognizing emotions elicited by autobiographical recollection. Human-computer interaction, 28(6), 479-517. https://doi.org/10.1080/07370024.2012.755421
    Janssen, J.H. ; Tacken, P. ; de Vries, G.J. ; van den Broek, Egon ; Westerink, J.H.D.M. ; Haselager, P. ; IJsselsteijn, W.A. / Machines outperform lay persons in recognizing emotions elicited by autobiographical recollection. In: Human-computer interaction. 2013 ; Vol. 28, No. 6. pp. 479-517.
    @article{29caeeec9a4c44b1bf800ab978e65282,
    title = "Machines outperform lay persons in recognizing emotions elicited by autobiographical recollection",
    abstract = "Over the last decade, an increasing number of studies have focused on automated recognition of human emotions by machines. However, performances of machine emotion recognition studies are difficult to interpret because benchmarks have not been established. In order to provide such a benchmark, we compared machine with human emotion recognition. We gathered facial expressions, speech, and physiological signals from 17 individuals expressing 5 different emotional states. Support vector machines achieved an 82{\%} recognition accuracy based on a physiological and facial features. In experiments with 75 humans on the same data, a maximum recognition accuracy of 62.8{\%} was obtained. As machines outperformed humans, automated emotion recognition might be ready to be tested in more practical applications.",
    keywords = "EWI-21692, IR-87031, HMI-CI: Computational Intelligence, HMI-SLT: Speech and Language Technology, HMI-MI: MULTIMODAL INTERACTIONS, Vision, Speech, Physiology, machine, autobiographical recollection, (Human), BioSignals, Affect, Machine Learning, Audio, Multimodal, Facial expressions, Emotion, METIS-297587, HMI-HF: Human Factors",
    author = "J.H. Janssen and P. Tacken and {de Vries}, G.J. and {van den Broek}, Egon and J.H.D.M. Westerink and P. Haselager and W.A. IJsselsteijn",
    note = "eemcs-eprint-21692",
    year = "2013",
    month = "7",
    day = "29",
    doi = "10.1080/07370024.2012.755421",
    language = "Undefined",
    volume = "28",
    pages = "479--517",
    journal = "Human-computer interaction",
    issn = "0737-0024",
    publisher = "Taylor & Francis",
    number = "6",

    }

    Janssen, JH, Tacken, P, de Vries, GJ, van den Broek, E, Westerink, JHDM, Haselager, P & IJsselsteijn, WA 2013, 'Machines outperform lay persons in recognizing emotions elicited by autobiographical recollection', Human-computer interaction, vol. 28, no. 6, pp. 479-517. https://doi.org/10.1080/07370024.2012.755421

    Machines outperform lay persons in recognizing emotions elicited by autobiographical recollection. / Janssen, J.H.; Tacken, P.; de Vries, G.J.; van den Broek, Egon; Westerink, J.H.D.M.; Haselager, P.; IJsselsteijn, W.A.

    In: Human-computer interaction, Vol. 28, No. 6, 29.07.2013, p. 479-517.

    Research output: Contribution to journalArticleAcademicpeer-review

    TY - JOUR

    T1 - Machines outperform lay persons in recognizing emotions elicited by autobiographical recollection

    AU - Janssen, J.H.

    AU - Tacken, P.

    AU - de Vries, G.J.

    AU - van den Broek, Egon

    AU - Westerink, J.H.D.M.

    AU - Haselager, P.

    AU - IJsselsteijn, W.A.

    N1 - eemcs-eprint-21692

    PY - 2013/7/29

    Y1 - 2013/7/29

    N2 - Over the last decade, an increasing number of studies have focused on automated recognition of human emotions by machines. However, performances of machine emotion recognition studies are difficult to interpret because benchmarks have not been established. In order to provide such a benchmark, we compared machine with human emotion recognition. We gathered facial expressions, speech, and physiological signals from 17 individuals expressing 5 different emotional states. Support vector machines achieved an 82% recognition accuracy based on a physiological and facial features. In experiments with 75 humans on the same data, a maximum recognition accuracy of 62.8% was obtained. As machines outperformed humans, automated emotion recognition might be ready to be tested in more practical applications.

    AB - Over the last decade, an increasing number of studies have focused on automated recognition of human emotions by machines. However, performances of machine emotion recognition studies are difficult to interpret because benchmarks have not been established. In order to provide such a benchmark, we compared machine with human emotion recognition. We gathered facial expressions, speech, and physiological signals from 17 individuals expressing 5 different emotional states. Support vector machines achieved an 82% recognition accuracy based on a physiological and facial features. In experiments with 75 humans on the same data, a maximum recognition accuracy of 62.8% was obtained. As machines outperformed humans, automated emotion recognition might be ready to be tested in more practical applications.

    KW - EWI-21692

    KW - IR-87031

    KW - HMI-CI: Computational Intelligence

    KW - HMI-SLT: Speech and Language Technology

    KW - HMI-MI: MULTIMODAL INTERACTIONS

    KW - Vision

    KW - Speech

    KW - Physiology

    KW - machine

    KW - autobiographical recollection

    KW - (Human)

    KW - BioSignals

    KW - Affect

    KW - Machine Learning

    KW - Audio

    KW - Multimodal

    KW - Facial expressions

    KW - Emotion

    KW - METIS-297587

    KW - HMI-HF: Human Factors

    U2 - 10.1080/07370024.2012.755421

    DO - 10.1080/07370024.2012.755421

    M3 - Article

    VL - 28

    SP - 479

    EP - 517

    JO - Human-computer interaction

    JF - Human-computer interaction

    SN - 0737-0024

    IS - 6

    ER -

    Janssen JH, Tacken P, de Vries GJ, van den Broek E, Westerink JHDM, Haselager P et al. Machines outperform lay persons in recognizing emotions elicited by autobiographical recollection. Human-computer interaction. 2013 Jul 29;28(6):479-517. https://doi.org/10.1080/07370024.2012.755421