Meta-Analysis of the First Facial Expression Recognition Challenge

M.F. Valstar, M. Mehu, Bihan Jiang, Maja Pantic, K. Scherer

    Research output: Contribution to journalArticleAcademicpeer-review

    249 Citations (Scopus)
    3 Downloads (Pure)


    Automatic facial expression recognition has been an active topic in computer science for over two decades, in particular facial action coding system action unit (AU) detection and classification of a number of discrete emotion states from facial expressive imagery. Standardization and comparability have received some attention; for instance, there exist a number of commonly used facial expression databases. However, lack of a commonly accepted evaluation protocol and, typically, lack of sufficient details needed to reproduce the reported individual results make it difficult to compare systems. This, in turn, hinders the progress of the field. A periodical challenge in facial expression recognition would allow such a comparison on a level playing field. It would provide an insight on how far the field has come and would allow researchers to identify new goals, challenges, and targets. This paper presents a meta-analysis of the first such challenge in automatic recognition of facial expressions, held during the IEEE conference on Face and Gesture Recognition 2011. It details the challenge data, evaluation protocol, and the results attained in two subchallenges: AU detection and classification of facial expression imagery in terms of a number of discrete emotion categories. We also summarize the lessons learned and reflect on the future of the field of facial expression recognition in general and on possible future challenges in particular.
    Original languageUndefined
    Pages (from-to)966-979
    Number of pages14
    JournalIEEE transactions on systems, man, and cybernetics. Part B: Cybernetics
    Issue number4
    Publication statusPublished - Aug 2012


    • facial action coding system (FACS) analysis
    • discrete emotion recognition
    • Challenges
    • Facial Expression Analysis
    • METIS-296208
    • IR-84213
    • EWI-22868

    Cite this