Social Signal Processing: Understanding Social Actions through nonverbal behaviour analysis

A. Vinciarelli, H. Salamin, Maja Pantic

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    52 Citations (Scopus)
    487 Downloads (Pure)


    This paper introduces social signal processing (SSP), the domain aimed at automatic understanding of social interactions through analysis of nonverbal behavior. The core idea of SSP is that nonverbal behavior is machine detectable evidence of social signals, the relational attitudes exchanged between interacting individuals. Social signals include (dis-)agreement, empathy, hostility, and any other attitude towards others that is expressed not only by words but by nonverbal behaviors such as facial expression and body posture as well. Thus, nonverbal behavior analysis is used as a key to automatic understanding of social interactions. This paper presents not only a survey of the related literature and the main concepts underlying SSP, but also an illustrative example of how such concepts are applied to the analysis of conflicts in competitive discussions.
    Original languageUndefined
    Title of host publicationIEEE International Conference on Computer Vision and Pattern Recognition (CVPR'09)
    Place of PublicationLos Alamitos
    Number of pages8
    ISBN (Print)978-1-4244-3994-2
    Publication statusPublished - 18 Aug 2009
    EventIEEE International Conference on Computer Vision and Pattern Recognition, CVPR '09 - Miami, FL, USA
    Duration: 20 Jun 200925 Jun 2009

    Publication series

    PublisherIEEE Computer Society Press


    ConferenceIEEE International Conference on Computer Vision and Pattern Recognition, CVPR '09
    Other20-25 June 2009


    • METIS-264327
    • Body posture
    • IR-69562
    • Nonverbal Behavior Analysis
    • Facial Expression
    • HMI-HF: Human Factors
    • EC Grant Agreement nr.: FP7/231287
    • social interactions
    • machine detectable evidence
    • Social Signal Processing
    • EWI-17216

    Cite this