Towards visual and vocal mimicry recognition in human-human interactions

X. Sun, Khiet Phuong Truong, Maja Pantic, Antinus Nijholt

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    24 Citations (Scopus)

    Abstract

    During face-to-face interpersonal interaction, people have a tendency to mimic each other. People not only mimic postures, mannerisms, moods or emotions, but they also mimic several speech-related behaviors. In this paper we describe how visual and vocal behavioral information expressed between two interlocutors can be used to detect and identify visual and vocal mimicry. We investigate expressions of mimicry and aim to learn more about in which situation and to what extent mimicry occurs. The observable effects of mimicry can be explored by representing and recognizing mimicry using visual and vocal features. In order to automatically analyze how to extract and integrate this behavioral information into a multimodal mimicry detection framework for improving affective computing, this paper addresses the main challenge: mimicry representation in terms of optimal behavioral feature extraction and automatic integration in both audio and video modalities.
    Original languageEnglish
    Title of host publicationIEEE SMC 2011
    Subtitle of host publication2011 IEEE International Conference on Systems, Man and Cybernetics October 9-12, 2011
    EditorsE. Tunstel, S. Nahavandi, A. Stoica
    Place of PublicationPiscataway, NJ
    PublisherIEEE Computer Society
    Pages367-373
    Number of pages7
    ISBN (Print)978-1-4577-0652-3
    DOIs
    Publication statusPublished - 21 Nov 2011
    EventIEEE International Conference on Systems, Man, and Cybernetics, SMC 2011 - Hilton Anchorage, Anchorage, United States
    Duration: 9 Oct 201112 Oct 2011

    Conference

    ConferenceIEEE International Conference on Systems, Man, and Cybernetics, SMC 2011
    Abbreviated titleSMC
    CountryUnited States
    CityAnchorage
    Period9/10/1112/10/11

    Keywords

    • METIS-281573
    • Human behavior analysis
    • Mimicry representation
    • IR-78613
    • human- human interaction
    • vocal mimicry
    • Visual mimicry
    • HMI-MI: MULTIMODAL INTERACTIONS
    • EWI-20822
    • EC Grant Agreement nr.: FP7/231287
    • EC Grant Agreement nr.: ERC/203143
    • interpersonal synchrony

    Fingerprint Dive into the research topics of 'Towards visual and vocal mimicry recognition in human-human interactions'. Together they form a unique fingerprint.

    Cite this