Social Signals, their function, and automatic analysis: A survey

Alessandro Vinciarelli, Maja Pantic, Hervé Bourlard, Alex Pentland

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    59 Citations (Scopus)


    Social Signal Processing (SSP) aims at the analysis of social behaviour in both Human-Human and Human-Computer interactions. SSP revolves around automatic sensing and interpretation of social signals, complex aggregates of nonverbal behaviours through which individuals express their attitudes towards other human (and virtual) participants in the current social context. As such, SSP integrates both engineering (speech analysis, computer vision, etc.) and human sciences (social psychology, anthropology, etc.) as it requires multimodal and multidisciplinary approaches. As of today, SSP is still in its early infancy, but the domain is quickly developing, and a growing number of works is appearing in the literature. This paper provides an introduction to nonverbal behaviour involved in social signals and a survey of the main results obtained so far in SSP. It also outlines possibilities and challenges that SSP is expected to face in the next years if it is to reach its full maturity.
    Original languageUndefined
    Title of host publicationProceedings of the 10th international conference on Multimodal interfaces (ICMI'08)
    Place of PublicationNew York
    PublisherAssociation for Computing Machinery (ACM)
    Number of pages8
    ISBN (Print)978-1-60558-198-9
    Publication statusPublished - Oct 2008
    Event10th International Conference on Multimodal Interfaces, ICMI 2008 - Chania, Crete, Greece
    Duration: 20 Oct 200822 Oct 2008
    Conference number: 10

    Publication series



    Conference10th International Conference on Multimodal Interfaces, ICMI 2008
    Abbreviated titleICMI
    CityChania, Crete


    • Speech Analysis
    • Social Behaviour Analysis
    • Social Signal Processing
    • METIS-255092
    • EC Grant Agreement nr.: FP6/0027787
    • EC Grant Agreement nr.: FP7/211486
    • IR-62674
    • Computer Vision
    • EWI-14817

    Cite this