Decision Level Fusion of Domain Specific Regions for Facial Action Recognition

Bihan Jiang, Brais Martinez, Michel F. Valstar, Maja Pantic

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    38 Citations (Scopus)
    46 Downloads (Pure)


    In this paper we propose a new method for the detection of action units that relies on a novel region-based face representation and a mid-level decision layer that combines region-specific information. Different from other approaches, we do not represent the face as a regular grid based on the face location alone (holistic representation), nor by using small patches centred at iducial facial point locations (local representation). Instead, we propose to use domain knowledge regarding AU-specific facial muscle contractions to define a set of face regions covering the whole face. Therefore, as opposed to local appearance models, our face representation makes use of the full facial appearance, while the use of facial point locations to define the regions means that we obtain better-registered descriptors compared to holistic representations. Finally, we propose an AU-specific weighted sum model is used as a decision-level fusion layer in charge of combining region-specific probabilistic information. This configuration allows each classier to learning the typical appearance changes for a specific face part and reduces the dimensionality of the problem thus proving to be more robust. Our approach is evaluated on the DISFA and GEMEP-FERA datasets using two histogram-based appearance features, Local Binary Pattern and Local Phase Quantisation. We show superior performance for both the domain-specific region definition and the decision-level fusion respect to the standard approaches when it comes to automatic facial action unit detection.
    Original languageUndefined
    Title of host publicationProceedings of the 22nd International Conference on Pattern Recognition (ICPR 2014)
    Place of PublicationUSA
    PublisherIEEE Computer Society
    Number of pages6
    ISBN (Print)978-1-4799-5208-3
    Publication statusPublished - Aug 2014
    Event22nd International Conference on Pattern Recognition 2014 - Stockholm, Sweden
    Duration: 24 Aug 201428 Aug 2014
    Conference number: 22

    Publication series

    PublisherIEEE Computer Society
    ISSN (Print)1051-4651


    Conference22nd International Conference on Pattern Recognition 2014
    Abbreviated titleICPR 2014
    Internet address


    • HMI-HF: Human Factors
    • EWI-25828
    • METIS-309953
    • EC Grant Agreement nr.: FP7/611153
    • IR-95234
    • EC Grant Agreement nr.: FP7/2007-2013

    Cite this