Copula Ordinal Regression for Joint Estimation of Facial Action Unit Intensity

Robert Walecki, Ognjen Rudovic, Vladimir Pavlovic, Maja Pantic

    Research output: Contribution to conferencePaperpeer-review

    35 Citations (Scopus)
    16 Downloads (Pure)


    Joint modeling of the intensity of facial action units (AUs) from face images is challenging due to the large number of AUs (30+) and their intensity levels (6). This is in part due to the lack of suitable models that can efficiently handle such a large number of outputs/classes simultaneously, but also due to the lack of labelled target data. For this reason, majority of the methods proposed so far resort to independent classifiers for the AU intensity. This is suboptimal for at least two reasons: the facial appearance of some AUs changes depending on the intensity of other AUs, and some AUs co-occur more often than others. Encoding this is expected to improve the estimation of target AU intensities, especially in the case of noisy image features, head-pose variations and imbalanced training data. To this end, we introduce a novel modeling framework, Copula Ordinal Regression (COR), that leverages the power of copula functions and CRFs, to detangle the probabilistic modeling of AU dependencies from the marginal modeling of the AU intensity. Consequently, the COR model achieves the joint learning and inference of intensities of multiple AUs, while being computationally tractable. We show on two challenging datasets of naturalistic facial expressions that the proposed approach consistently outperforms (i) independent modeling of AU intensities, and (ii) the state-ofthe-art approach for the target task.
    Original languageUndefined
    Number of pages9
    Publication statusPublished - 26 Jun 2016
    Event29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016 - Las Vegas, NV, USA, Las Vegas, United States
    Duration: 26 Jun 20161 Jul 2016
    Conference number: 29


    Conference29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
    Abbreviated titleCVPR 2016
    Country/TerritoryUnited States
    CityLas Vegas
    Other29 June - 1 July 2016


    • IR-104071
    • HMI-HF: Human Factors
    • EWI-27131
    • facial action units
    • Copula Ordinal Regression (COR)
    • Human facial expressions

    Cite this