Designing Frameworks for Automatic Affect Prediction and Classi﬿cation in Dimensional Space

Mihalis A. Nicolaou, Hatice Gunes, Maja Pantic

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    2 Citations (Scopus)

    Abstract

    This paper focuses on designing frameworks for automatic affect prediction and classification in dimensional space. Similarly to many pattern recognition problems, dimensional affect prediction requires predicting multidimensional output vectors (e.g., valence and arousal) given a specific set of input features (e.g., facial expression cues). To date, affect recognition in valence and arousal space has been done separately along each dimension, assuming that they are independent. However, various psychological findings suggest that these dimensions are correlated. In light of this, we focus on modeling inter-dimensional correlations, and propose (i) an Output-Associative Relevance Vector Machine (OA-RVM) regression framework that augments the traditional RVM regression by being able to learn non-linear input and output dependencies among affect dimensions, and (ii) a multi-layer hybrid framework composed of a temporal regression layer for predicting affect dimensions, a graphical model layer for modeling valence-arousal correlations, and a final classification and fusion layer exploiting informative statistics extracted from the lower layers. We demonstrate the effectiveness and the robustness of the proposed frameworks by subject-independent experimental validation(s) performed on a naturalistic data set of facial expressions.
    Original languageUndefined
    Title of host publicationIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2011)
    Place of PublicationUSA
    PublisherIEEE
    Pages20-26
    Number of pages7
    ISBN (Print)978-1-4577-0529-8
    DOIs
    Publication statusPublished - Jun 2011
    Event24th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011 - Colorado Springs, United States
    Duration: 20 Jun 201125 Jun 2011
    Conference number: 24

    Publication series

    Name
    PublisherIEEE Computer Society
    ISSN (Print)2160-7508

    Conference

    Conference24th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011
    Abbreviated titleCVPR 2011
    Country/TerritoryUnited States
    CityColorado Springs
    Period20/06/1125/06/11

    Keywords

    • METIS-285033
    • IR-79501
    • Emotion Recognition
    • Support Vector Machines
    • HMI-MI: MULTIMODAL INTERACTIONS
    • Regression analysis
    • EC Grant Agreement nr.: FP7/231287
    • EWI-21333
    • Pattern Recognition

    Cite this