Affective Signal Processing (ASP): Unraveling the mystery of emotions

Egon van den Broek

    Research output: ThesisPhD Thesis - Research UT, graduation UT

    1566 Downloads (Pure)


    Slowly computers are being dressed and becoming huggable and tangible. They are being personalized and are expected to understand more of their users' feelings, emotions, and moods: This we refer to as affective computing. The work and experiences from 50+ publications on affective computing is collected and reported in one concise monograph. A brief introduction on emotion theory and affective computing is given and its relevance for computer science (i.e., Human-Computer Interaction, Artificial Intelligence (AI), and Health Informatics) is denoted. Next, a closed model for affective computing is introduced and reviews of both biosignals and affective computing are presented. The conclusion of all of this is that affective computing lacks standards. Affective computing's key dimensions need to be identified and studied to bring the field the progress it needs. A series of studies is presented that explore baseline-free affective computing, the influence of a range of distinct triggers for emotions, the influence of time windows, several combinations of biosignals with and without speech, the impact of the context, the personality traits neuroticism and extroversion, and demographics. Moreover, a complete signal processing + classification processing pipeline for affective computing is developed and executed on the data gathered. This pipeline is also applied on two clinical case studies, which proves that affective computing is already feasible in clinical practice; that is, Computer Aided Diagnosis (CAD) for patients suffering from a Post-Traumatic Stress Disorder (PTSD). A set of key dimensions for affective computing and, subsequently, prerequisites and guidelines are defined, based on the author's work and experiences. Moreover, historical, emotion theoretical, and computer science perspectives are taken to reflect upon the work and five applications (i.e., next-generation TV, knowledge representations, CAD, robot nannies, and digital human models) are presented to illustrate the field's future impact. If anything, this monograph shows that affective computing will become unstoppable, will determine our relation with ICT and, as such, will reshape our lives.
    Original languageEnglish
    QualificationDoctor of Philosophy
    Awarding Institution
    • University of Twente
    • Nijholt, Anton, Supervisor
    • Dijkstra, T., Supervisor, External person
    • Westerink, J.H.D.M., Advisor, External person
    Award date16 Sep 2011
    Place of PublicationEnschede
    Print ISBNs978-90-365-3243-3
    Publication statusPublished - 16 Sep 2011


    • METIS-281568
    • Emotion
    • HMI-HF: Human Factors
    • HMI-CI: Computational Intelligence
    • HMI-SLT: Speech and Language Technology
    • HMI-IE: Information Engineering
    • ASP
    • EWI-20812
    • psychophysiology
    • Pattern Recognition
    • Statistics
    • Prerequisites
    • Validity
    • Validation
    • Speech
    • overview
    • HMI-IA: Intelligent Agents
    • Modeling
    • Guidelines
    • Review
    • Experimentation
    • Affective Signal Processing
    • Affective Computing
    • Affect
    • Machine Learning
    • IR-78025
    • BioSignals


    Dive into the research topics of 'Affective Signal Processing (ASP): Unraveling the mystery of emotions'. Together they form a unique fingerprint.

    Cite this