A Framework for Joint Estimation and Guided Annotation of Facial Action Unit Intensity

Robert Walecki, Ognjen Rudovic, Maja Pantic, Vladimir Pavlovic, Jeffrey F. Cohn

    Research output: Contribution to conferencePaperAcademicpeer-review

    4 Citations (Scopus)
    67 Downloads (Pure)


    Manual annotation of facial action units (AUs) is highly tedious and time-consuming. Various methods for automatic coding of AUs have been proposed, however, their performance is still far below of that attained by expert human coders. Several attempts have been made to leverage these methods to reduce the burden of manual coding of AU activations (presence/absence). Nevertheless, this has not been exploited in the context of AU intensity coding, which is a far more difficult task. To this end, we propose an expertdriven probabilistic approach for joint modeling and estimation of AU intensities. Specifically, we introduce a Conditional Random Field model for joint estimation of the AU intensity that updates its predictions in an iterative fashion by relying on expert knowledge of human coders. We show in our experiments on two publicly available datasets of AU intensity (DISFA and FERA2015) that the AU coding process can significantly be facilitated by the proposed approach, allowing human coders to faster make decisions about target AU intensity.
    Original languageUndefined
    Number of pages9
    Publication statusPublished - 26 Jun 2016
    Event29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016 - Las Vegas, NV, USA, Las Vegas, United States
    Duration: 26 Jun 20161 Jul 2016
    Conference number: 29


    Conference29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
    Abbreviated titleCVPR 2016
    Country/TerritoryUnited States
    CityLas Vegas
    Other29 June - 1 July 2016


    • HMI-HF: Human Factors
    • human coders
    • EC Grant Agreement nr.: FP7/645094
    • facial action units
    • EWI-27134
    • EC Grant Agreement nr.: FP7/688835
    • IR-104106

    Cite this