Designing Frameworks for Automatic Affect Prediction and Classi﬿cation in Dimensional Space

This paper focuses on designing frameworks for automatic affect prediction and classification in dimensional space. Similarly to many pattern recognition problems, dimensional affect prediction requires predicting multidimensional output vectors (e.g., valence and arousal) given a specific set of input features (e.g., facial expression cues). To date, affect recognition in valence and arousal space has been done separately along each dimension, assuming that they are independent. However, various psychological findings suggest that these dimensions are correlated. In light of this, we focus on modeling inter-dimensional correlations, and propose (i) an Output-Associative Relevance Vector Machine (OA-RVM) regression framework that augments the traditional RVM regression by being able to learn non-linear input and output dependencies among affect dimensions, and (ii) a multi-layer hybrid framework composed of a temporal regression layer for predicting affect dimensions, a graphical model layer for modeling valence-arousal correlations, and a final classification and fusion layer exploiting informative statistics extracted from the lower layers. We demonstrate the effectiveness and the robustness of the proposed frameworks by subject-independent experimental validation(s) performed on a naturalistic data set of facial expressions.