A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions

Zhihong Zeng, Maja Pantic, Glenn I. Roisman, Thomas S. Huang

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    66 Citations (Scopus)
    1038 Downloads (Pure)


    Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. Promising approaches have been reported, including automatic methods for facial and vocal affect recognition. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions-despite the fact that deliberate behavior differs in visual and audio expressions from spontaneously occurring behavior. Recently efforts to develop algorithms that can process naturally occurring human affective behavior have emerged. This paper surveys these efforts. We first discuss human emotion perception from a psychological perspective. Next, we examine the available approaches to solving the problem of machine understanding of human affective behavior occurring in real-world settings. We finally outline some scientific and engineering challenges for advancing human affect sensing technology.
    Original languageEnglish
    Title of host publicationICMI '07
    Subtitle of host publicationproceedings of the Ninth International Conference on Multimodal Interfaces, Nagoya, Aichi, Japan, November 12-15, 2007
    Place of PublicationNew York, NY, USA
    PublisherAssociation for Computing Machinery (ACM)
    Number of pages8
    ISBN (Print)978-1-59593-817-6
    Publication statusPublished - 14 Nov 2007
    Event9th International Conference on Multimodal Interfaces, ICMI 2007 - Nagoya, Japan
    Duration: 12 Nov 200715 Nov 2007
    Conference number: 9


    Conference9th International Conference on Multimodal Interfaces, ICMI 2007
    Abbreviated titleICMI


    • Multimodal human computer interaction
    • Multimodal user interfaces
    • Affective computing
    • Human computing
    • Affect recognition
    • Emotion recognition


    Dive into the research topics of 'A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions'. Together they form a unique fingerprint.

    Cite this