Social Signals, their function, and automatic analysis: A survey

Alessandro Vinciarelli; Maja Pantic; Hervé Bourlard; Alex Pentland

doi:10.1145/1452392.1452405

Social Signals, their function, and automatic analysis: A survey

Alessandro Vinciarelli, Maja Pantic, Hervé Bourlard, Alex Pentland

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

54 Citations (Scopus)

Abstract

Social Signal Processing (SSP) aims at the analysis of social behaviour in both Human-Human and Human-Computer interactions. SSP revolves around automatic sensing and interpretation of social signals, complex aggregates of nonverbal behaviours through which individuals express their attitudes towards other human (and virtual) participants in the current social context. As such, SSP integrates both engineering (speech analysis, computer vision, etc.) and human sciences (social psychology, anthropology, etc.) as it requires multimodal and multidisciplinary approaches. As of today, SSP is still in its early infancy, but the domain is quickly developing, and a growing number of works is appearing in the literature. This paper provides an introduction to nonverbal behaviour involved in social signals and a survey of the main results obtained so far in SSP. It also outlines possibilities and challenges that SSP is expected to face in the next years if it is to reach its full maturity.

Original language	Undefined
Title of host publication	Proceedings of the 10th international conference on Multimodal interfaces (ICMI'08)
Place of Publication	New York
Publisher	Association for Computing Machinery (ACM)
Pages	61-68
Number of pages	8
ISBN (Print)	978-1-60558-198-9
DOIs	https://doi.org/10.1145/1452392.1452405
Publication status	Published - Oct 2008
Event	10th International Conference on Multimodal Interfaces, ICMI 2008 - Chania, Crete, Greece Duration: 20 Oct 2008 → 22 Oct 2008 Conference number: 10

Publication series

Name
Publisher	ACM
Number	2008/16200

Conference

Conference	10th International Conference on Multimodal Interfaces, ICMI 2008
Abbreviated title	ICMI
Country	Greece
City	Chania, Crete
Period	20/10/08 → 22/10/08

Keywords

HMI-MI: MULTIMODAL INTERACTIONS
Speech Analysis
Social Behaviour Analysis
Social Signal Processing
METIS-255092
EC Grant Agreement nr.: FP6/0027787
EC Grant Agreement nr.: FP7/211486
IR-62674
Computer Vision
EWI-14817

Access to Document

10.1145/1452392.1452405

http://eprints.eemcs.utwente.nl/secure2/14817/01/pantic_social_signals_icmi08.pdf

Cite this

@inproceedings{ae41d75a5fe84ee69e3f2342b421a296,

title = "Social Signals, their function, and automatic analysis: A survey",

abstract = "Social Signal Processing (SSP) aims at the analysis of social behaviour in both Human-Human and Human-Computer interactions. SSP revolves around automatic sensing and interpretation of social signals, complex aggregates of nonverbal behaviours through which individuals express their attitudes towards other human (and virtual) participants in the current social context. As such, SSP integrates both engineering (speech analysis, computer vision, etc.) and human sciences (social psychology, anthropology, etc.) as it requires multimodal and multidisciplinary approaches. As of today, SSP is still in its early infancy, but the domain is quickly developing, and a growing number of works is appearing in the literature. This paper provides an introduction to nonverbal behaviour involved in social signals and a survey of the main results obtained so far in SSP. It also outlines possibilities and challenges that SSP is expected to face in the next years if it is to reach its full maturity.",

keywords = "HMI-MI: MULTIMODAL INTERACTIONS, Speech Analysis, Social Behaviour Analysis, Social Signal Processing, METIS-255092, EC Grant Agreement nr.: FP6/0027787, EC Grant Agreement nr.: FP7/211486, IR-62674, Computer Vision, EWI-14817",

author = "Alessandro Vinciarelli and Maja Pantic and Herv{\'e} Bourlard and Alex Pentland",

note = "10.1145/1452392.1452405 ; null ; Conference date: 20-10-2008 Through 22-10-2008",

year = "2008",

month = oct,

doi = "10.1145/1452392.1452405",

language = "Undefined",

isbn = "978-1-60558-198-9",

publisher = "Association for Computing Machinery (ACM)",

number = "2008/16200",

pages = "61--68",

booktitle = "Proceedings of the 10th international conference on Multimodal interfaces (ICMI'08)",

address = "United States",

}

Vinciarelli, A, Pantic, M, Bourlard, H & Pentland, A 2008, Social Signals, their function, and automatic analysis: A survey. in Proceedings of the 10th international conference on Multimodal interfaces (ICMI'08)., 10.1145/1452392.1452405, Association for Computing Machinery (ACM), New York, pp. 61-68, 10th International Conference on Multimodal Interfaces, ICMI 2008, Chania, Crete, Greece, 20/10/08. https://doi.org/10.1145/1452392.1452405

Social Signals, their function, and automatic analysis: A survey. / Vinciarelli, Alessandro; Pantic, Maja; Bourlard, Hervé; Pentland, Alex.

Proceedings of the 10th international conference on Multimodal interfaces (ICMI'08). New York : Association for Computing Machinery (ACM), 2008. p. 61-68 10.1145/1452392.1452405.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Social Signals, their function, and automatic analysis: A survey

AU - Vinciarelli, Alessandro

AU - Pantic, Maja

AU - Bourlard, Hervé

AU - Pentland, Alex

N1 - Conference code: 10

PY - 2008/10

Y1 - 2008/10

N2 - Social Signal Processing (SSP) aims at the analysis of social behaviour in both Human-Human and Human-Computer interactions. SSP revolves around automatic sensing and interpretation of social signals, complex aggregates of nonverbal behaviours through which individuals express their attitudes towards other human (and virtual) participants in the current social context. As such, SSP integrates both engineering (speech analysis, computer vision, etc.) and human sciences (social psychology, anthropology, etc.) as it requires multimodal and multidisciplinary approaches. As of today, SSP is still in its early infancy, but the domain is quickly developing, and a growing number of works is appearing in the literature. This paper provides an introduction to nonverbal behaviour involved in social signals and a survey of the main results obtained so far in SSP. It also outlines possibilities and challenges that SSP is expected to face in the next years if it is to reach its full maturity.

AB - Social Signal Processing (SSP) aims at the analysis of social behaviour in both Human-Human and Human-Computer interactions. SSP revolves around automatic sensing and interpretation of social signals, complex aggregates of nonverbal behaviours through which individuals express their attitudes towards other human (and virtual) participants in the current social context. As such, SSP integrates both engineering (speech analysis, computer vision, etc.) and human sciences (social psychology, anthropology, etc.) as it requires multimodal and multidisciplinary approaches. As of today, SSP is still in its early infancy, but the domain is quickly developing, and a growing number of works is appearing in the literature. This paper provides an introduction to nonverbal behaviour involved in social signals and a survey of the main results obtained so far in SSP. It also outlines possibilities and challenges that SSP is expected to face in the next years if it is to reach its full maturity.

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - Speech Analysis

KW - Social Behaviour Analysis

KW - Social Signal Processing

KW - METIS-255092

KW - EC Grant Agreement nr.: FP6/0027787

KW - EC Grant Agreement nr.: FP7/211486

KW - IR-62674

KW - Computer Vision

KW - EWI-14817

U2 - 10.1145/1452392.1452405

DO - 10.1145/1452392.1452405

M3 - Conference contribution

SN - 978-1-60558-198-9

SP - 61

EP - 68

BT - Proceedings of the 10th international conference on Multimodal interfaces (ICMI'08)

PB - Association for Computing Machinery (ACM)

CY - New York

Y2 - 20 October 2008 through 22 October 2008

ER -