Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation

Sebastian Kaltwang, Sinisa Todorovic, Maja Pantic

    Research output: Contribution to journalArticleAcademicpeer-review

    42 Citations (Scopus)


    Certain inner feelings and physiological states like pain are subjective states that cannot be directly measured, but can be estimated from spontaneous facial expressions. Since they are typically characterized by subtle movements of facial parts, analysis of the facial details is required. To this end, we formulate a new regression method for continuous estimation of the intensity of facial behavior interpretation, called Doubly Sparse Relevance Vector Machine (DSRVM). DSRVM enforces double sparsity by jointly selecting the most relevant training examples (a.k.a. relevance vectors) and the most important kernels associated with facial parts relevant for interpretation of observed facial expressions. This advances prior work on multi-kernel learning, where sparsity of relevant kernels is typically ignored. Empirical evaluation on challenging Shoulder Pain videos, and the benchmark DISFA and SEMAINE datasets demonstrate that DSRVM outperforms competing approaches with a multi-fold reduction of running times in training and testing.
    Original languageEnglish
    Pages (from-to)1748-1761
    Number of pages18
    JournalIEEE transactions on pattern analysis and machine intelligence
    Issue number9
    Publication statusPublished - Sep 2016


    • HMI-HF: Human Factors
    • EWI-26749
    • Facial expressions
    • METIS-315563
    • Relevance Vector Machine
    • IR-99333
    • Multiple Kernel Learning
    • Regression
    • n/a OA procedure


    Dive into the research topics of 'Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation'. Together they form a unique fingerprint.

    Cite this