Correlated-Spaces Regression for Learning Continuous Emotion Dimensions

M. Nicolaou, S. Zafeiriou, Maja Pantic

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    17 Citations (Scopus)
    69 Downloads (Pure)

    Abstract

    Adopting continuous dimensional annotations for affective analysis has been gaining rising attention by researchers over the past years. Due to the idiosyncratic nature of this problem, many subproblems have been identified, spanning from the fusion of multiple continuous annotations to exploiting output-correlations amongst emotion dimensions. In this paper, we firstly empirically answer several important questions which have found partial or no answer at all so far in related literature. In more detail, we study the correlation of each emotion dimension (i) with respect to other emotion dimensions, (ii) to basic emotions (e.g., happiness, anger). As a measure for comparison, we use video and audio features. Interestingly enough, we find that (i) each emotion dimension is more correlated with other emotion dimensions rather than with face and audio features, and similarly (ii) that each basic emotion is more correlated with emotion dimensions than with audio and video features. A similar conclusion holds for discrete emotions which are found to be highly correlated to emotion dimensions as compared to audio and/or video features. Motivated by these findings, we present a novel regression algorithm (Correlated-Spaces Regression, CSR), inspired by Canonical Correlation Analysis (CCA) which learns output-correlations and performs supervised dimensionality reduction and multimodal fusion by (i) projecting features extracted from all modalities and labels onto a common space where their inter-correlation is maximised and (ii) learning mappings from the projected feature space onto the projected, uncorrelated label space.
    Original languageUndefined
    Title of host publicationProceedings of the 21st ACM international conference on Multimedia, MM 2013
    Place of PublicationNew York
    PublisherAssociation for Computing Machinery (ACM)
    Pages773-776
    Number of pages4
    ISBN (Print)978-1-4503-2404-5
    DOIs
    Publication statusPublished - Oct 2013
    Event21st ACM Multimedia Conference, MM 2013 - Barcelona, Spain
    Duration: 21 Oct 201325 Oct 2013
    Conference number: 21
    http://acmmm13.org/general-info/about-acm-multimedia-2013/

    Publication series

    Name
    PublisherACM

    Conference

    Conference21st ACM Multimedia Conference, MM 2013
    Abbreviated titleMM
    CountrySpain
    CityBarcelona
    Period21/10/1325/10/13
    Internet address

    Keywords

    • EWI-24345
    • HMI-HF: Human Factors
    • output-correlations
    • valence
    • EC Grant Agreement nr.: FP7/2007-2013
    • EC Grant Agreement nr.: FP7/288235
    • IR-89375
    • Continuous and dimensional emotion descriptions
    • Arousal
    • Component Analysis
    • Multi-modal Fusion
    • Feature Selection
    • METIS-302664
    • EC Grant Agreement nr.: ERC-2007-STG-203143 (MAHNOB)

    Cite this