Communicative Signals and Social Contextual Factors in Multimodal Affect Recognition

Michel-Pierre Jansen

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

2 Citations (Scopus)
34 Downloads (Pure)


One research branch in Affective Computing focuses on using multimodal ‘emotional’ expressions (e.g. facial expressions or non-verbal vocalisations) to automatically detect emotions and affect experienced by persons. The field is increasingly interested in using contextual factors to better infer emotional expressions rather than solely relying on the emotional expressions by themselves. We are interested in expressions that occur in a social context. In our research we plan to investigate how we can; a) utilise communicative signals that are displayed during interactions to recognise social contextual factors that influence emotion expression and in turn b) predict/recognise what these emotion expressions are most likely communicating considering the context. To achieve this, we formulate three main research questions: I) How do communicative signals such as emotion expressions co-ordinate behaviours and knowledge between interlocutors in interactive settings?, II) Can we use behavioural cues during interactions to detect social contextual factors relevant for interpreting affect? and III) Can we use social contextual factors and communicative signals to predict what emotion experience is linked to an emotion expression?
Original languageEnglish
Title of host publicationICMI '19
Subtitle of host publication2019 International Conference on Multimodal Interaction
EditorsWen Gao, Helen Mei Ling Meng, Matthew Turk, Susan R. Fussell
PublisherAssociation for Computing Machinery
Number of pages4
ISBN (Print)978-1-4503-6860-5
Publication statusPublished - 15 Oct 2019
Event21st ACM International Conference on Multimodal Interaction, ICMI 2019 - Worldhotel Grand Dushulake Suzhou, Suzhou, China
Duration: 14 Oct 201918 Oct 2019
Conference number: 21


Conference21st ACM International Conference on Multimodal Interaction, ICMI 2019
Abbreviated titleICMI 2019
Internet address


Dive into the research topics of 'Communicative Signals and Social Contextual Factors in Multimodal Affect Recognition'. Together they form a unique fingerprint.

Cite this