Towards visual and vocal mimicry recognition in human-human interactions

X. Sun; Khiet Phuong Truong; Maja Pantic; Antinus Nijholt

doi:10.1109/ICSMC.2011.6083693

Towards visual and vocal mimicry recognition in human-human interactions

X. Sun, Khiet Phuong Truong, Maja Pantic, Antinus Nijholt

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

27 Citations (Scopus)

15 Downloads (Pure)

Abstract

During face-to-face interpersonal interaction, people have a tendency to mimic each other. People not only mimic postures, mannerisms, moods or emotions, but they also mimic several speech-related behaviors. In this paper we describe how visual and vocal behavioral information expressed between two interlocutors can be used to detect and identify visual and vocal mimicry. We investigate expressions of mimicry and aim to learn more about in which situation and to what extent mimicry occurs. The observable effects of mimicry can be explored by representing and recognizing mimicry using visual and vocal features. In order to automatically analyze how to extract and integrate this behavioral information into a multimodal mimicry detection framework for improving affective computing, this paper addresses the main challenge: mimicry representation in terms of optimal behavioral feature extraction and automatic integration in both audio and video modalities.

Original language	English
Title of host publication	IEEE SMC 2011
Subtitle of host publication	2011 IEEE International Conference on Systems, Man and Cybernetics October 9-12, 2011
Editors	E. Tunstel, S. Nahavandi, A. Stoica
Place of Publication	Piscataway, NJ
Publisher	IEEE
Pages	367-373
Number of pages	7
ISBN (Print)	978-1-4577-0652-3
DOIs	https://doi.org/10.1109/ICSMC.2011.6083693
Publication status	Published - 21 Nov 2011
Event	IEEE International Conference on Systems, Man, and Cybernetics, SMC 2011 - Hilton Anchorage, Anchorage, United States Duration: 9 Oct 2011 → 12 Oct 2011

Conference

Conference	IEEE International Conference on Systems, Man, and Cybernetics, SMC 2011
Abbreviated title	SMC
Country/Territory	United States
City	Anchorage
Period	9/10/11 → 12/10/11

Keywords

METIS-281573
Human behavior analysis
Mimicry representation
IR-78613
human- human interaction
vocal mimicry
Visual mimicry
HMI-MI: MULTIMODAL INTERACTIONS
EWI-20822
EC Grant Agreement nr.: FP7/231287
EC Grant Agreement nr.: ERC/203143
interpersonal synchrony

Access to Document

10.1109/ICSMC.2011.6083693

http://eprints.eemcs.utwente.nl/secure2/20822/01/smc2011.pdf

Cite this

@inproceedings{ae9c2c79e0e349bbbf3b281282336682,

title = "Towards visual and vocal mimicry recognition in human-human interactions",

abstract = "During face-to-face interpersonal interaction, people have a tendency to mimic each other. People not only mimic postures, mannerisms, moods or emotions, but they also mimic several speech-related behaviors. In this paper we describe how visual and vocal behavioral information expressed between two interlocutors can be used to detect and identify visual and vocal mimicry. We investigate expressions of mimicry and aim to learn more about in which situation and to what extent mimicry occurs. The observable effects of mimicry can be explored by representing and recognizing mimicry using visual and vocal features. In order to automatically analyze how to extract and integrate this behavioral information into a multimodal mimicry detection framework for improving affective computing, this paper addresses the main challenge: mimicry representation in terms of optimal behavioral feature extraction and automatic integration in both audio and video modalities.",

keywords = "METIS-281573, Human behavior analysis, Mimicry representation, IR-78613, human- human interaction, vocal mimicry, Visual mimicry, HMI-MI: MULTIMODAL INTERACTIONS, EWI-20822, EC Grant Agreement nr.: FP7/231287, EC Grant Agreement nr.: ERC/203143, interpersonal synchrony",

author = "X. Sun and Truong, {Khiet Phuong} and Maja Pantic and Antinus Nijholt",

note = "Special Session on Social Signal Processing ; IEEE International Conference on Systems, Man, and Cybernetics, SMC 2011, SMC ; Conference date: 09-10-2011 Through 12-10-2011",

year = "2011",

month = nov,

day = "21",

doi = "10.1109/ICSMC.2011.6083693",

language = "English",

isbn = "978-1-4577-0652-3",

pages = "367--373",

editor = "E. Tunstel and S. Nahavandi and A. Stoica",

booktitle = "IEEE SMC 2011",

publisher = "IEEE",

address = "United States",

}

Sun, X, Truong, KP, Pantic, M & Nijholt, A 2011, Towards visual and vocal mimicry recognition in human-human interactions. in E Tunstel, S Nahavandi & A Stoica (eds), IEEE SMC 2011: 2011 IEEE International Conference on Systems, Man and Cybernetics October 9-12, 2011. IEEE, Piscataway, NJ, pp. 367-373, IEEE International Conference on Systems, Man, and Cybernetics, SMC 2011, Anchorage, Alaska, United States, 9/10/11. https://doi.org/10.1109/ICSMC.2011.6083693

Towards visual and vocal mimicry recognition in human-human interactions. / Sun, X.; Truong, Khiet Phuong; Pantic, Maja et al.
IEEE SMC 2011: 2011 IEEE International Conference on Systems, Man and Cybernetics October 9-12, 2011. ed. / E. Tunstel; S. Nahavandi; A. Stoica. Piscataway, NJ: IEEE, 2011. p. 367-373.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Towards visual and vocal mimicry recognition in human-human interactions

AU - Sun, X.

AU - Truong, Khiet Phuong

AU - Pantic, Maja

AU - Nijholt, Antinus

N1 - Special Session on Social Signal Processing

PY - 2011/11/21

Y1 - 2011/11/21

N2 - During face-to-face interpersonal interaction, people have a tendency to mimic each other. People not only mimic postures, mannerisms, moods or emotions, but they also mimic several speech-related behaviors. In this paper we describe how visual and vocal behavioral information expressed between two interlocutors can be used to detect and identify visual and vocal mimicry. We investigate expressions of mimicry and aim to learn more about in which situation and to what extent mimicry occurs. The observable effects of mimicry can be explored by representing and recognizing mimicry using visual and vocal features. In order to automatically analyze how to extract and integrate this behavioral information into a multimodal mimicry detection framework for improving affective computing, this paper addresses the main challenge: mimicry representation in terms of optimal behavioral feature extraction and automatic integration in both audio and video modalities.

AB - During face-to-face interpersonal interaction, people have a tendency to mimic each other. People not only mimic postures, mannerisms, moods or emotions, but they also mimic several speech-related behaviors. In this paper we describe how visual and vocal behavioral information expressed between two interlocutors can be used to detect and identify visual and vocal mimicry. We investigate expressions of mimicry and aim to learn more about in which situation and to what extent mimicry occurs. The observable effects of mimicry can be explored by representing and recognizing mimicry using visual and vocal features. In order to automatically analyze how to extract and integrate this behavioral information into a multimodal mimicry detection framework for improving affective computing, this paper addresses the main challenge: mimicry representation in terms of optimal behavioral feature extraction and automatic integration in both audio and video modalities.

KW - METIS-281573

KW - Human behavior analysis

KW - Mimicry representation

KW - IR-78613

KW - human- human interaction

KW - vocal mimicry

KW - Visual mimicry

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - EWI-20822

KW - EC Grant Agreement nr.: FP7/231287

KW - EC Grant Agreement nr.: ERC/203143

KW - interpersonal synchrony

U2 - 10.1109/ICSMC.2011.6083693

DO - 10.1109/ICSMC.2011.6083693

M3 - Conference contribution

SN - 978-1-4577-0652-3

SP - 367

EP - 373

BT - IEEE SMC 2011

A2 - Tunstel, E.

A2 - Nahavandi, S.

A2 - Stoica, A.

PB - IEEE

CY - Piscataway, NJ

T2 - IEEE International Conference on Systems, Man, and Cybernetics, SMC 2011

Y2 - 9 October 2011 through 12 October 2011

ER -

Towards visual and vocal mimicry recognition in human-human interactions

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this