Social Signal Processing: Understanding Social Actions through nonverbal behaviour analysis

A. Vinciarelli, H. Salamin, Maja Pantic

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

36 Citations (Scopus)
87 Downloads (Pure)

Abstract

This paper introduces social signal processing (SSP), the domain aimed at automatic understanding of social interactions through analysis of nonverbal behavior. The core idea of SSP is that nonverbal behavior is machine detectable evidence of social signals, the relational attitudes exchanged between interacting individuals. Social signals include (dis-)agreement, empathy, hostility, and any other attitude towards others that is expressed not only by words but by nonverbal behaviors such as facial expression and body posture as well. Thus, nonverbal behavior analysis is used as a key to automatic understanding of social interactions. This paper presents not only a survey of the related literature and the main concepts underlying SSP, but also an illustrative example of how such concepts are applied to the analysis of conflicts in competitive discussions.
Original languageUndefined
Title of host publicationIEEE International Conference on Computer Vision and Pattern Recognition (CVPR'09)
Place of PublicationLos Alamitos
PublisherIEEE Computer Society
Pages42-49
Number of pages8
ISBN (Print)978-1-4244-3994-2
DOIs
Publication statusPublished - 18 Aug 2009

Publication series

Name
PublisherIEEE Computer Society Press
Volume3

Keywords

  • METIS-264327
  • Body posture
  • IR-69562
  • Nonverbal Behavior Analysis
  • Facial Expression
  • HMI-HF: Human Factors
  • EC Grant Agreement nr.: FP7/231287
  • social interactions
  • machine detectable evidence
  • Social Signal Processing
  • HMI-MI: MULTIMODAL INTERACTIONS
  • EWI-17216

Cite this

Vinciarelli, A., Salamin, H., & Pantic, M. (2009). Social Signal Processing: Understanding Social Actions through nonverbal behaviour analysis. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'09) (pp. 42-49). [10.1109/CVPR.2009.5204290] Los Alamitos: IEEE Computer Society. https://doi.org/10.1109/CVPR.2009.5204290
Vinciarelli, A. ; Salamin, H. ; Pantic, Maja. / Social Signal Processing: Understanding Social Actions through nonverbal behaviour analysis. IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'09). Los Alamitos : IEEE Computer Society, 2009. pp. 42-49
@inproceedings{51e02dc4db9849dc9f9cf34f90ee4ff7,
title = "Social Signal Processing: Understanding Social Actions through nonverbal behaviour analysis",
abstract = "This paper introduces social signal processing (SSP), the domain aimed at automatic understanding of social interactions through analysis of nonverbal behavior. The core idea of SSP is that nonverbal behavior is machine detectable evidence of social signals, the relational attitudes exchanged between interacting individuals. Social signals include (dis-)agreement, empathy, hostility, and any other attitude towards others that is expressed not only by words but by nonverbal behaviors such as facial expression and body posture as well. Thus, nonverbal behavior analysis is used as a key to automatic understanding of social interactions. This paper presents not only a survey of the related literature and the main concepts underlying SSP, but also an illustrative example of how such concepts are applied to the analysis of conflicts in competitive discussions.",
keywords = "METIS-264327, Body posture, IR-69562, Nonverbal Behavior Analysis, Facial Expression, HMI-HF: Human Factors, EC Grant Agreement nr.: FP7/231287, social interactions, machine detectable evidence, Social Signal Processing, HMI-MI: MULTIMODAL INTERACTIONS, EWI-17216",
author = "A. Vinciarelli and H. Salamin and Maja Pantic",
note = "10.1109/CVPR.2009.5204290",
year = "2009",
month = "8",
day = "18",
doi = "10.1109/CVPR.2009.5204290",
language = "Undefined",
isbn = "978-1-4244-3994-2",
publisher = "IEEE Computer Society",
pages = "42--49",
booktitle = "IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'09)",
address = "United States",

}

Vinciarelli, A, Salamin, H & Pantic, M 2009, Social Signal Processing: Understanding Social Actions through nonverbal behaviour analysis. in IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'09)., 10.1109/CVPR.2009.5204290, IEEE Computer Society, Los Alamitos, pp. 42-49. https://doi.org/10.1109/CVPR.2009.5204290

Social Signal Processing: Understanding Social Actions through nonverbal behaviour analysis. / Vinciarelli, A.; Salamin, H.; Pantic, Maja.

IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'09). Los Alamitos : IEEE Computer Society, 2009. p. 42-49 10.1109/CVPR.2009.5204290.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Social Signal Processing: Understanding Social Actions through nonverbal behaviour analysis

AU - Vinciarelli, A.

AU - Salamin, H.

AU - Pantic, Maja

N1 - 10.1109/CVPR.2009.5204290

PY - 2009/8/18

Y1 - 2009/8/18

N2 - This paper introduces social signal processing (SSP), the domain aimed at automatic understanding of social interactions through analysis of nonverbal behavior. The core idea of SSP is that nonverbal behavior is machine detectable evidence of social signals, the relational attitudes exchanged between interacting individuals. Social signals include (dis-)agreement, empathy, hostility, and any other attitude towards others that is expressed not only by words but by nonverbal behaviors such as facial expression and body posture as well. Thus, nonverbal behavior analysis is used as a key to automatic understanding of social interactions. This paper presents not only a survey of the related literature and the main concepts underlying SSP, but also an illustrative example of how such concepts are applied to the analysis of conflicts in competitive discussions.

AB - This paper introduces social signal processing (SSP), the domain aimed at automatic understanding of social interactions through analysis of nonverbal behavior. The core idea of SSP is that nonverbal behavior is machine detectable evidence of social signals, the relational attitudes exchanged between interacting individuals. Social signals include (dis-)agreement, empathy, hostility, and any other attitude towards others that is expressed not only by words but by nonverbal behaviors such as facial expression and body posture as well. Thus, nonverbal behavior analysis is used as a key to automatic understanding of social interactions. This paper presents not only a survey of the related literature and the main concepts underlying SSP, but also an illustrative example of how such concepts are applied to the analysis of conflicts in competitive discussions.

KW - METIS-264327

KW - Body posture

KW - IR-69562

KW - Nonverbal Behavior Analysis

KW - Facial Expression

KW - HMI-HF: Human Factors

KW - EC Grant Agreement nr.: FP7/231287

KW - social interactions

KW - machine detectable evidence

KW - Social Signal Processing

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - EWI-17216

U2 - 10.1109/CVPR.2009.5204290

DO - 10.1109/CVPR.2009.5204290

M3 - Conference contribution

SN - 978-1-4244-3994-2

SP - 42

EP - 49

BT - IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'09)

PB - IEEE Computer Society

CY - Los Alamitos

ER -

Vinciarelli A, Salamin H, Pantic M. Social Signal Processing: Understanding Social Actions through nonverbal behaviour analysis. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR'09). Los Alamitos: IEEE Computer Society. 2009. p. 42-49. 10.1109/CVPR.2009.5204290 https://doi.org/10.1109/CVPR.2009.5204290