Context in Human Emotion Perception for Automatic Affect Detection: A Survey of Audiovisual Databases

Bernd Dudzik, Michel-Pierre Jansen, Franziska Burger, Frank Kaptein, Joost Broekens, Dirk K.J. Heylen, Hayley Hung, Mark Neerincx, Khiet Phuong Truong

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)

Abstract

An important aspect of human emotion perception is the use of contextual information to understand others' feelings even in situations where their behavior is not very expressive or has an emotionally ambiguous meaning. For technology to successfully detect affect, it must mimic this human ability when analyzing audiovisual input. Databases upon which machine learning algorithms are trained should capture the context of social interactions as well as the behavior expressed in them. However, there is a lack of consensus about what constitutes relevant context in such databases. In this article, we make two contributions towards overcoming this challenge: (a) we identify two principal sources of context for emotion perceptions based on psychological theory, and (b) we provide an overview of how each of these has been considered in published databases covering social interactions. Our results show that a similar set of contextual features are present across the reviewed databases. Between all the different databases researchers seem to have taken into account a set of contextual features reflecting the sources of context seen in psychological theory. However, within individual databases, these features are not yet systematically varied. This is problematic because it prevents them from being used directly as resources for the modeling of context-sensitive affect detection. Based on our findings, we suggest improvements for the future development of affective databases.
Original languageEnglish
Title of host publication2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)
Place of PublicationPiscataway, NJ
PublisherIEEE
Pages206-212
ISBN (Electronic)978-1-7281-3888-6
ISBN (Print)978-1-7281-3889-3
DOIs
Publication statusPublished - Dec 2019
Event8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019 - Cambridge, United Kingdom
Duration: 3 Sep 20196 Sep 2019
Conference number: 8
http://acii-conf.org/2019/

Publication series

NameInternational Conference on Affective Computing and Intelligent Interaction (ACII)
PublisherIEEE
Volume2019
ISSN (Print)2156-8103
ISSN (Electronic)2156-8111

Conference

Conference8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019
Abbreviated titleACII
CountryUnited Kingdom
CityCambridge
Period3/09/196/09/19
Internet address

Keywords

  • Context
  • Automatic affect detection
  • Human emotion perception
  • Audiovisual database
  • Survey

Fingerprint Dive into the research topics of 'Context in Human Emotion Perception for Automatic Affect Detection: A Survey of Audiovisual Databases'. Together they form a unique fingerprint.

  • Cite this

    Dudzik, B., Jansen, M-P., Burger, F., Kaptein, F., Broekens, J., Heylen, D. K. J., ... Truong, K. P. (2019). Context in Human Emotion Perception for Automatic Affect Detection: A Survey of Audiovisual Databases. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 206-212). (International Conference on Affective Computing and Intelligent Interaction (ACII); Vol. 2019). Piscataway, NJ: IEEE. https://doi.org/10.1109/ACII.2019.8925446