DEAP: A Database for Emotion Analysis Using Physiological Signals

Sander Koelstra, C. Mühl, Mohammad Soleymani, Jung Seok Lee, Ashkan Yazdani, Touradj Ebrahimi, Thierry Pun, Antinus Nijholt, Ioannis Patras

  • 456 Citations

Abstract

We present a multimodal dataset for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence and like/dislike ratings using the modalities of EEG, peripheral physiological signals and multimedia content analysis. Finally, decision fusion of the classification results from the different modalities is performed. The dataset is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.
Original languageUndefined
Pages (from-to)18-31
Number of pages15
JournalIEEE transactions on affective computing
Volume3
Issue number1
DOIs
StatePublished - 2012

Fingerprint

Electroencephalography
State estimation
Websites
Fusion reactions
Testing
Experiments

Keywords

  • Affect sensing and analysis
  • Machine Learning
  • Multimedia Retrieval
  • Music retrieval and generation
  • Methods for emotion elicitation
  • IR-79512
  • Emotion in human-computer interaction
  • Emotional Corpora
  • Physiological Measures
  • EWI-21368
  • METIS-285051
  • Methods of data collection

Cite this

Koelstra, S., Mühl, C., Soleymani, M., Lee, J. S., Yazdani, A., Ebrahimi, T., ... Patras, I. (2012). DEAP: A Database for Emotion Analysis Using Physiological Signals. IEEE transactions on affective computing, 3(1), 18-31. DOI: 10.1109/T-AFFC.2011.15

Koelstra, Sander; Mühl, C.; Soleymani, Mohammad; Lee, Jung Seok; Yazdani, Ashkan; Ebrahimi, Touradj; Pun, Thierry; Nijholt, Antinus; Patras, Ioannis / DEAP: A Database for Emotion Analysis Using Physiological Signals.

In: IEEE transactions on affective computing, Vol. 3, No. 1, 2012, p. 18-31.

Research output: Scientific - peer-reviewArticle

@article{16d2153c76974234a7af4ed017b3bb68,
title = "DEAP: A Database for Emotion Analysis Using Physiological Signals",
abstract = "We present a multimodal dataset for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence and like/dislike ratings using the modalities of EEG, peripheral physiological signals and multimedia content analysis. Finally, decision fusion of the classification results from the different modalities is performed. The dataset is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.",
keywords = "Affect sensing and analysis, Machine Learning, Multimedia Retrieval, Music retrieval and generation, Methods for emotion elicitation, IR-79512, Emotion in human-computer interaction, Emotional Corpora, Physiological Measures, EWI-21368, METIS-285051, Methods of data collection",
author = "Sander Koelstra and C. Mühl and Mohammad Soleymani and Lee, {Jung Seok} and Ashkan Yazdani and Touradj Ebrahimi and Thierry Pun and Antinus Nijholt and Ioannis Patras",
note = "eemcs-eprint-21368",
year = "2012",
doi = "10.1109/T-AFFC.2011.15",
volume = "3",
pages = "18--31",
journal = "IEEE transactions on affective computing",
issn = "1949-3045",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "1",

}

Koelstra, S, Mühl, C, Soleymani, M, Lee, JS, Yazdani, A, Ebrahimi, T, Pun, T, Nijholt, A & Patras, I 2012, 'DEAP: A Database for Emotion Analysis Using Physiological Signals' IEEE transactions on affective computing, vol 3, no. 1, pp. 18-31. DOI: 10.1109/T-AFFC.2011.15

DEAP: A Database for Emotion Analysis Using Physiological Signals. / Koelstra, Sander; Mühl, C.; Soleymani, Mohammad; Lee, Jung Seok; Yazdani, Ashkan; Ebrahimi, Touradj; Pun, Thierry; Nijholt, Antinus; Patras, Ioannis.

In: IEEE transactions on affective computing, Vol. 3, No. 1, 2012, p. 18-31.

Research output: Scientific - peer-reviewArticle

TY - JOUR

T1 - DEAP: A Database for Emotion Analysis Using Physiological Signals

AU - Koelstra,Sander

AU - Mühl,C.

AU - Soleymani,Mohammad

AU - Lee,Jung Seok

AU - Yazdani,Ashkan

AU - Ebrahimi,Touradj

AU - Pun,Thierry

AU - Nijholt,Antinus

AU - Patras,Ioannis

N1 - eemcs-eprint-21368

PY - 2012

Y1 - 2012

N2 - We present a multimodal dataset for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence and like/dislike ratings using the modalities of EEG, peripheral physiological signals and multimedia content analysis. Finally, decision fusion of the classification results from the different modalities is performed. The dataset is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.

AB - We present a multimodal dataset for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence and like/dislike ratings using the modalities of EEG, peripheral physiological signals and multimedia content analysis. Finally, decision fusion of the classification results from the different modalities is performed. The dataset is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.

KW - Affect sensing and analysis

KW - Machine Learning

KW - Multimedia Retrieval

KW - Music retrieval and generation

KW - Methods for emotion elicitation

KW - IR-79512

KW - Emotion in human-computer interaction

KW - Emotional Corpora

KW - Physiological Measures

KW - EWI-21368

KW - METIS-285051

KW - Methods of data collection

U2 - 10.1109/T-AFFC.2011.15

DO - 10.1109/T-AFFC.2011.15

M3 - Article

VL - 3

SP - 18

EP - 31

JO - IEEE transactions on affective computing

T2 - IEEE transactions on affective computing

JF - IEEE transactions on affective computing

SN - 1949-3045

IS - 1

ER -

Koelstra S, Mühl C, Soleymani M, Lee JS, Yazdani A, Ebrahimi T et al. DEAP: A Database for Emotion Analysis Using Physiological Signals. IEEE transactions on affective computing. 2012;3(1):18-31. Available from, DOI: 10.1109/T-AFFC.2011.15