Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation

Sebastian Kaltwang, Sinisa Todorovic, Maja Pantic

  • 4 Citations

Abstract

Certain inner feelings and physiological states like pain are subjective states that cannot be directly measured, but can be estimated from spontaneous facial expressions. Since they are typically characterized by subtle movements of facial parts, analysis of the facial details is required. To this end, we formulate a new regression method for continuous estimation of the intensity of facial behavior interpretation, called Doubly Sparse Relevance Vector Machine (DSRVM). DSRVM enforces double sparsity by jointly selecting the most relevant training examples (a.k.a. relevance vectors) and the most important kernels associated with facial parts relevant for interpretation of observed facial expressions. This advances prior work on multi-kernel learning, where sparsity of relevant kernels is typically ignored. Empirical evaluation on challenging Shoulder Pain videos, and the benchmark DISFA and SEMAINE datasets demonstrate that DSRVM outperforms competing approaches with a multi-fold reduction of running times in training and testing.
Original languageUndefined
Pages (from-to)1748-1761
Number of pages18
JournalIEEE transactions on pattern analysis and machine intelligence
Volume38
Issue number9
DOIs
StatePublished - Sep 2016

Fingerprint

Testing

Keywords

  • HMI-HF: Human Factors
  • EWI-26749
  • Facial expressions
  • METIS-315563
  • Relevance Vector Machine
  • IR-99333
  • Multiple Kernel Learning
  • Regression

Cite this

Kaltwang, Sebastian; Todorovic, Sinisa; Pantic, Maja / Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation.

In: IEEE transactions on pattern analysis and machine intelligence, Vol. 38, No. 9, 09.2016, p. 1748-1761.

Research output: Scientific - peer-reviewArticle

@article{75b9fed6934c4ada98da86dee8c734c2,
title = "Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation",
abstract = "Certain inner feelings and physiological states like pain are subjective states that cannot be directly measured, but can be estimated from spontaneous facial expressions. Since they are typically characterized by subtle movements of facial parts, analysis of the facial details is required. To this end, we formulate a new regression method for continuous estimation of the intensity of facial behavior interpretation, called Doubly Sparse Relevance Vector Machine (DSRVM). DSRVM enforces double sparsity by jointly selecting the most relevant training examples (a.k.a. relevance vectors) and the most important kernels associated with facial parts relevant for interpretation of observed facial expressions. This advances prior work on multi-kernel learning, where sparsity of relevant kernels is typically ignored. Empirical evaluation on challenging Shoulder Pain videos, and the benchmark DISFA and SEMAINE datasets demonstrate that DSRVM outperforms competing approaches with a multi-fold reduction of running times in training and testing.",
keywords = "HMI-HF: Human Factors, EWI-26749, Facial expressions, METIS-315563, Relevance Vector Machine, IR-99333, Multiple Kernel Learning, Regression",
author = "Sebastian Kaltwang and Sinisa Todorovic and Maja Pantic",
note = "eemcs-eprint-2674",
year = "2016",
month = "9",
doi = "10.1109/TPAMI.2015.2501824",
volume = "38",
pages = "1748--1761",
journal = "IEEE transactions on pattern analysis and machine intelligence",
issn = "0162-8828",
publisher = "IEEE Computer Society",
number = "9",

}

Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation. / Kaltwang, Sebastian; Todorovic, Sinisa; Pantic, Maja.

In: IEEE transactions on pattern analysis and machine intelligence, Vol. 38, No. 9, 09.2016, p. 1748-1761.

Research output: Scientific - peer-reviewArticle

TY - JOUR

T1 - Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation

AU - Kaltwang,Sebastian

AU - Todorovic,Sinisa

AU - Pantic,Maja

N1 - eemcs-eprint-2674

PY - 2016/9

Y1 - 2016/9

N2 - Certain inner feelings and physiological states like pain are subjective states that cannot be directly measured, but can be estimated from spontaneous facial expressions. Since they are typically characterized by subtle movements of facial parts, analysis of the facial details is required. To this end, we formulate a new regression method for continuous estimation of the intensity of facial behavior interpretation, called Doubly Sparse Relevance Vector Machine (DSRVM). DSRVM enforces double sparsity by jointly selecting the most relevant training examples (a.k.a. relevance vectors) and the most important kernels associated with facial parts relevant for interpretation of observed facial expressions. This advances prior work on multi-kernel learning, where sparsity of relevant kernels is typically ignored. Empirical evaluation on challenging Shoulder Pain videos, and the benchmark DISFA and SEMAINE datasets demonstrate that DSRVM outperforms competing approaches with a multi-fold reduction of running times in training and testing.

AB - Certain inner feelings and physiological states like pain are subjective states that cannot be directly measured, but can be estimated from spontaneous facial expressions. Since they are typically characterized by subtle movements of facial parts, analysis of the facial details is required. To this end, we formulate a new regression method for continuous estimation of the intensity of facial behavior interpretation, called Doubly Sparse Relevance Vector Machine (DSRVM). DSRVM enforces double sparsity by jointly selecting the most relevant training examples (a.k.a. relevance vectors) and the most important kernels associated with facial parts relevant for interpretation of observed facial expressions. This advances prior work on multi-kernel learning, where sparsity of relevant kernels is typically ignored. Empirical evaluation on challenging Shoulder Pain videos, and the benchmark DISFA and SEMAINE datasets demonstrate that DSRVM outperforms competing approaches with a multi-fold reduction of running times in training and testing.

KW - HMI-HF: Human Factors

KW - EWI-26749

KW - Facial expressions

KW - METIS-315563

KW - Relevance Vector Machine

KW - IR-99333

KW - Multiple Kernel Learning

KW - Regression

U2 - 10.1109/TPAMI.2015.2501824

DO - 10.1109/TPAMI.2015.2501824

M3 - Article

VL - 38

SP - 1748

EP - 1761

JO - IEEE transactions on pattern analysis and machine intelligence

T2 - IEEE transactions on pattern analysis and machine intelligence

JF - IEEE transactions on pattern analysis and machine intelligence

SN - 0162-8828

IS - 9

ER -