Affective Signal Processing (ASP): Unraveling the mystery of emotions

Egon van den Broek

Abstract

Slowly computers are being dressed and becoming huggable and tangible. They are being personalized and are expected to understand more of their users' feelings, emotions, and moods: This we refer to as affective computing. The work and experiences from 50+ publications on affective computing is collected and reported in one concise monograph. A brief introduction on emotion theory and affective computing is given and its relevance for computer science (i.e., Human-Computer Interaction, Artificial Intelligence (AI), and Health Informatics) is denoted. Next, a closed model for affective computing is introduced and reviews of both biosignals and affective computing are presented. The conclusion of all of this is that affective computing lacks standards. Affective computing's key dimensions need to be identified and studied to bring the field the progress it needs. A series of studies is presented that explore baseline-free affective computing, the influence of a range of distinct triggers for emotions, the influence of time windows, several combinations of biosignals with and without speech, the impact of the context, the personality traits neuroticism and extroversion, and demographics. Moreover, a complete signal processing + classification processing pipeline for affective computing is developed and executed on the data gathered. This pipeline is also applied on two clinical case studies, which proves that affective computing is already feasible in clinical practice; that is, Computer Aided Diagnosis (CAD) for patients suffering from a Post-Traumatic Stress Disorder (PTSD). A set of key dimensions for affective computing and, subsequently, prerequisites and guidelines are defined, based on the author's work and experiences. Moreover, historical, emotion theoretical, and computer science perspectives are taken to reflect upon the work and five applications (i.e., next-generation TV, knowledge representations, CAD, robot nannies, and digital human models) are presented to illustrate the field's future impact. If anything, this monograph shows that affective computing will become unstoppable, will determine our relation with ICT and, as such, will reshape our lives.
Original languageEnglish
Awarding Institution
  • University of Twente
Supervisors/Advisors
  • Nijholt, Antinus , Supervisor
  • Dijkstra, T., Supervisor
  • Westerink, J.H.D.M., Advisor
Date of Award16 Sep 2011
Place of PublicationEnschede
Print ISBNs978-90-365-3243-3
DOIs
StatePublished - 16 Sep 2011

Fingerprint

Computer aided diagnosis
Computer science
Signal processing
Pipelines
Knowledge representation
Human computer interaction
Artificial intelligence
Health
Robots

Keywords

  • METIS-281568
  • Emotion
  • HMI-HF: Human Factors
  • HMI-CI: Computational Intelligence
  • HMI-SLT: Speech and Language Technology
  • HMI-MI: MULTIMODAL INTERACTIONS
  • HMI-IE: Information Engineering
  • ASP
  • EWI-20812
  • psychophysiology
  • Pattern Recognition
  • Statistics
  • Prerequisites
  • Validity
  • Validation
  • Speech
  • overview
  • HMI-IA: Intelligent Agents
  • Modeling
  • Guidelines
  • Review
  • Experimentation
  • Affective Signal Processing
  • Affective Computing
  • Affect
  • Machine Learning
  • IR-78025
  • BioSignals

Cite this

van den Broek, Egon. / Affective Signal Processing (ASP): Unraveling the mystery of emotions. Enschede, 2011. 302 p.
@misc{384cfc554a49461fa1a9936c2de00abc,
title = "Affective Signal Processing (ASP): Unraveling the mystery of emotions",
abstract = "Slowly computers are being dressed and becoming huggable and tangible. They are being personalized and are expected to understand more of their users' feelings, emotions, and moods: This we refer to as affective computing. The work and experiences from 50+ publications on affective computing is collected and reported in one concise monograph. A brief introduction on emotion theory and affective computing is given and its relevance for computer science (i.e., Human-Computer Interaction, Artificial Intelligence (AI), and Health Informatics) is denoted. Next, a closed model for affective computing is introduced and reviews of both biosignals and affective computing are presented. The conclusion of all of this is that affective computing lacks standards. Affective computing's key dimensions need to be identified and studied to bring the field the progress it needs. A series of studies is presented that explore baseline-free affective computing, the influence of a range of distinct triggers for emotions, the influence of time windows, several combinations of biosignals with and without speech, the impact of the context, the personality traits neuroticism and extroversion, and demographics. Moreover, a complete signal processing + classification processing pipeline for affective computing is developed and executed on the data gathered. This pipeline is also applied on two clinical case studies, which proves that affective computing is already feasible in clinical practice; that is, Computer Aided Diagnosis (CAD) for patients suffering from a Post-Traumatic Stress Disorder (PTSD). A set of key dimensions for affective computing and, subsequently, prerequisites and guidelines are defined, based on the author's work and experiences. Moreover, historical, emotion theoretical, and computer science perspectives are taken to reflect upon the work and five applications (i.e., next-generation TV, knowledge representations, CAD, robot nannies, and digital human models) are presented to illustrate the field's future impact. If anything, this monograph shows that affective computing will become unstoppable, will determine our relation with ICT and, as such, will reshape our lives.",
keywords = "METIS-281568, Emotion, HMI-HF: Human Factors, HMI-CI: Computational Intelligence, HMI-SLT: Speech and Language Technology, HMI-MI: MULTIMODAL INTERACTIONS, HMI-IE: Information Engineering, ASP, EWI-20812, psychophysiology, Pattern Recognition, Statistics, Prerequisites, Validity, Validation, Speech, overview, HMI-IA: Intelligent Agents, Modeling, Guidelines, Review, Experimentation, Affective Signal Processing, Affective Computing, Affect, Machine Learning, IR-78025, BioSignals",
author = "{van den Broek}, Egon",
note = "A review of this PhD thesis can be found at: Esposito, A. (2012). Affective Signal Processing (ASP): Unraveling the mystery of emotions, by Egon L. van den Broek. Journal of Ambient Intelligence and Smart Environments, 4(1), 75-76. DOI/URL: http://dx.doi.org/10.3233/AIS-2011-0131",
year = "2011",
month = "9",
doi = "10.3990/1.9789036532433",
isbn = "978-90-365-3243-3",
school = "University of Twente",

}

van den Broek, E 2011, 'Affective Signal Processing (ASP): Unraveling the mystery of emotions', University of Twente, Enschede. DOI: 10.3990/1.9789036532433

Affective Signal Processing (ASP): Unraveling the mystery of emotions. / van den Broek, Egon.

Enschede, 2011. 302 p.

Research output: ScientificPhD Thesis - Research UT, graduation UT

TY - THES

T1 - Affective Signal Processing (ASP): Unraveling the mystery of emotions

AU - van den Broek,Egon

N1 - A review of this PhD thesis can be found at: Esposito, A. (2012). Affective Signal Processing (ASP): Unraveling the mystery of emotions, by Egon L. van den Broek. Journal of Ambient Intelligence and Smart Environments, 4(1), 75-76. DOI/URL: http://dx.doi.org/10.3233/AIS-2011-0131

PY - 2011/9/16

Y1 - 2011/9/16

N2 - Slowly computers are being dressed and becoming huggable and tangible. They are being personalized and are expected to understand more of their users' feelings, emotions, and moods: This we refer to as affective computing. The work and experiences from 50+ publications on affective computing is collected and reported in one concise monograph. A brief introduction on emotion theory and affective computing is given and its relevance for computer science (i.e., Human-Computer Interaction, Artificial Intelligence (AI), and Health Informatics) is denoted. Next, a closed model for affective computing is introduced and reviews of both biosignals and affective computing are presented. The conclusion of all of this is that affective computing lacks standards. Affective computing's key dimensions need to be identified and studied to bring the field the progress it needs. A series of studies is presented that explore baseline-free affective computing, the influence of a range of distinct triggers for emotions, the influence of time windows, several combinations of biosignals with and without speech, the impact of the context, the personality traits neuroticism and extroversion, and demographics. Moreover, a complete signal processing + classification processing pipeline for affective computing is developed and executed on the data gathered. This pipeline is also applied on two clinical case studies, which proves that affective computing is already feasible in clinical practice; that is, Computer Aided Diagnosis (CAD) for patients suffering from a Post-Traumatic Stress Disorder (PTSD). A set of key dimensions for affective computing and, subsequently, prerequisites and guidelines are defined, based on the author's work and experiences. Moreover, historical, emotion theoretical, and computer science perspectives are taken to reflect upon the work and five applications (i.e., next-generation TV, knowledge representations, CAD, robot nannies, and digital human models) are presented to illustrate the field's future impact. If anything, this monograph shows that affective computing will become unstoppable, will determine our relation with ICT and, as such, will reshape our lives.

AB - Slowly computers are being dressed and becoming huggable and tangible. They are being personalized and are expected to understand more of their users' feelings, emotions, and moods: This we refer to as affective computing. The work and experiences from 50+ publications on affective computing is collected and reported in one concise monograph. A brief introduction on emotion theory and affective computing is given and its relevance for computer science (i.e., Human-Computer Interaction, Artificial Intelligence (AI), and Health Informatics) is denoted. Next, a closed model for affective computing is introduced and reviews of both biosignals and affective computing are presented. The conclusion of all of this is that affective computing lacks standards. Affective computing's key dimensions need to be identified and studied to bring the field the progress it needs. A series of studies is presented that explore baseline-free affective computing, the influence of a range of distinct triggers for emotions, the influence of time windows, several combinations of biosignals with and without speech, the impact of the context, the personality traits neuroticism and extroversion, and demographics. Moreover, a complete signal processing + classification processing pipeline for affective computing is developed and executed on the data gathered. This pipeline is also applied on two clinical case studies, which proves that affective computing is already feasible in clinical practice; that is, Computer Aided Diagnosis (CAD) for patients suffering from a Post-Traumatic Stress Disorder (PTSD). A set of key dimensions for affective computing and, subsequently, prerequisites and guidelines are defined, based on the author's work and experiences. Moreover, historical, emotion theoretical, and computer science perspectives are taken to reflect upon the work and five applications (i.e., next-generation TV, knowledge representations, CAD, robot nannies, and digital human models) are presented to illustrate the field's future impact. If anything, this monograph shows that affective computing will become unstoppable, will determine our relation with ICT and, as such, will reshape our lives.

KW - METIS-281568

KW - Emotion

KW - HMI-HF: Human Factors

KW - HMI-CI: Computational Intelligence

KW - HMI-SLT: Speech and Language Technology

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - HMI-IE: Information Engineering

KW - ASP

KW - EWI-20812

KW - psychophysiology

KW - Pattern Recognition

KW - Statistics

KW - Prerequisites

KW - Validity

KW - Validation

KW - Speech

KW - overview

KW - HMI-IA: Intelligent Agents

KW - Modeling

KW - Guidelines

KW - Review

KW - Experimentation

KW - Affective Signal Processing

KW - Affective Computing

KW - Affect

KW - Machine Learning

KW - IR-78025

KW - BioSignals

U2 - 10.3990/1.9789036532433

DO - 10.3990/1.9789036532433

M3 - PhD Thesis - Research UT, graduation UT

SN - 978-90-365-3243-3

ER -

van den Broek E. Affective Signal Processing (ASP): Unraveling the mystery of emotions. Enschede, 2011. 302 p. Available from, DOI: 10.3990/1.9789036532433