Coupled Gaussian processes for pose-invariant facial expression recognition

Ognjen Rudovic, Maja Pantic, Ioannis (Yannis) Patras

Research output: Contribution to journalArticleAcademicpeer-review

92 Citations (Scopus)

Abstract

We propose a method for head-pose invariant facial expression recognition that is based on a set of characteristic facial points. To achieve head-pose invariance, we propose the Coupled Scaled Gaussian Process Regression (CSGPR) model for head-pose normalization. In this model, we first learn independently the mappings between the facial points in each pair of (discrete) nonfrontal poses and the frontal pose, and then perform their coupling in order to capture dependences between them. During inference, the outputs of the coupled functions from different poses are combined using a gating function, devised based on the head-pose estimation for the query points. The proposed model outperforms state-of-the-art regression-based approaches to head-pose normalization, 2D and 3D Point Distribution Models (PDMs), and Active Appearance Models (AAMs), especially in cases of unknown poses and imbalanced training data. To the best of our knowledge, the proposed method is the first one that is able to deal with expressive faces in the range from -45° to +45° pan rotation and -30° to +30° tilt rotation, and with continuous changes in head pose, despite the fact that training was conducted on a small set of discrete poses. We evaluate the proposed method on synthetic and real images depicting acted and spontaneously displayed facial expressions.
Original languageUndefined
Pages (from-to)1357-1369
Number of pages13
JournalIEEE transactions on pattern analysis and machine intelligence
Volume35
Issue number6
DOIs
Publication statusPublished - Jun 2013

Keywords

  • Estimation
  • Face Recognition
  • Head
  • Multiview/pose-invariant facial expression/emotion recognition
  • Gaussian process regression
  • Magnetic heads
  • Active Appearance Model
  • IR-89546
  • METIS-302611
  • HMI-HF: Human Factors
  • EWI-24248
  • head-pose estimation
  • Solid modeling
  • Training

Cite this

Rudovic, Ognjen ; Pantic, Maja ; Patras, Ioannis (Yannis). / Coupled Gaussian processes for pose-invariant facial expression recognition. In: IEEE transactions on pattern analysis and machine intelligence. 2013 ; Vol. 35, No. 6. pp. 1357-1369.
@article{81abd81ab872442d92b229b2b53dfb0e,
title = "Coupled Gaussian processes for pose-invariant facial expression recognition",
abstract = "We propose a method for head-pose invariant facial expression recognition that is based on a set of characteristic facial points. To achieve head-pose invariance, we propose the Coupled Scaled Gaussian Process Regression (CSGPR) model for head-pose normalization. In this model, we first learn independently the mappings between the facial points in each pair of (discrete) nonfrontal poses and the frontal pose, and then perform their coupling in order to capture dependences between them. During inference, the outputs of the coupled functions from different poses are combined using a gating function, devised based on the head-pose estimation for the query points. The proposed model outperforms state-of-the-art regression-based approaches to head-pose normalization, 2D and 3D Point Distribution Models (PDMs), and Active Appearance Models (AAMs), especially in cases of unknown poses and imbalanced training data. To the best of our knowledge, the proposed method is the first one that is able to deal with expressive faces in the range from -45° to +45° pan rotation and -30° to +30° tilt rotation, and with continuous changes in head pose, despite the fact that training was conducted on a small set of discrete poses. We evaluate the proposed method on synthetic and real images depicting acted and spontaneously displayed facial expressions.",
keywords = "Estimation, Face Recognition, Head, Multiview/pose-invariant facial expression/emotion recognition, Gaussian process regression, Magnetic heads, Active Appearance Model, IR-89546, METIS-302611, HMI-HF: Human Factors, EWI-24248, head-pose estimation, Solid modeling, Training",
author = "Ognjen Rudovic and Maja Pantic and Patras, {Ioannis (Yannis)}",
note = "eemcs-eprint-24248",
year = "2013",
month = "6",
doi = "10.1109/TPAMI.2012.233",
language = "Undefined",
volume = "35",
pages = "1357--1369",
journal = "IEEE transactions on pattern analysis and machine intelligence",
issn = "0162-8828",
publisher = "IEEE Computer Society",
number = "6",

}

Coupled Gaussian processes for pose-invariant facial expression recognition. / Rudovic, Ognjen; Pantic, Maja; Patras, Ioannis (Yannis).

In: IEEE transactions on pattern analysis and machine intelligence, Vol. 35, No. 6, 06.2013, p. 1357-1369.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Coupled Gaussian processes for pose-invariant facial expression recognition

AU - Rudovic, Ognjen

AU - Pantic, Maja

AU - Patras, Ioannis (Yannis)

N1 - eemcs-eprint-24248

PY - 2013/6

Y1 - 2013/6

N2 - We propose a method for head-pose invariant facial expression recognition that is based on a set of characteristic facial points. To achieve head-pose invariance, we propose the Coupled Scaled Gaussian Process Regression (CSGPR) model for head-pose normalization. In this model, we first learn independently the mappings between the facial points in each pair of (discrete) nonfrontal poses and the frontal pose, and then perform their coupling in order to capture dependences between them. During inference, the outputs of the coupled functions from different poses are combined using a gating function, devised based on the head-pose estimation for the query points. The proposed model outperforms state-of-the-art regression-based approaches to head-pose normalization, 2D and 3D Point Distribution Models (PDMs), and Active Appearance Models (AAMs), especially in cases of unknown poses and imbalanced training data. To the best of our knowledge, the proposed method is the first one that is able to deal with expressive faces in the range from -45° to +45° pan rotation and -30° to +30° tilt rotation, and with continuous changes in head pose, despite the fact that training was conducted on a small set of discrete poses. We evaluate the proposed method on synthetic and real images depicting acted and spontaneously displayed facial expressions.

AB - We propose a method for head-pose invariant facial expression recognition that is based on a set of characteristic facial points. To achieve head-pose invariance, we propose the Coupled Scaled Gaussian Process Regression (CSGPR) model for head-pose normalization. In this model, we first learn independently the mappings between the facial points in each pair of (discrete) nonfrontal poses and the frontal pose, and then perform their coupling in order to capture dependences between them. During inference, the outputs of the coupled functions from different poses are combined using a gating function, devised based on the head-pose estimation for the query points. The proposed model outperforms state-of-the-art regression-based approaches to head-pose normalization, 2D and 3D Point Distribution Models (PDMs), and Active Appearance Models (AAMs), especially in cases of unknown poses and imbalanced training data. To the best of our knowledge, the proposed method is the first one that is able to deal with expressive faces in the range from -45° to +45° pan rotation and -30° to +30° tilt rotation, and with continuous changes in head pose, despite the fact that training was conducted on a small set of discrete poses. We evaluate the proposed method on synthetic and real images depicting acted and spontaneously displayed facial expressions.

KW - Estimation

KW - Face Recognition

KW - Head

KW - Multiview/pose-invariant facial expression/emotion recognition

KW - Gaussian process regression

KW - Magnetic heads

KW - Active Appearance Model

KW - IR-89546

KW - METIS-302611

KW - HMI-HF: Human Factors

KW - EWI-24248

KW - head-pose estimation

KW - Solid modeling

KW - Training

U2 - 10.1109/TPAMI.2012.233

DO - 10.1109/TPAMI.2012.233

M3 - Article

VL - 35

SP - 1357

EP - 1369

JO - IEEE transactions on pattern analysis and machine intelligence

JF - IEEE transactions on pattern analysis and machine intelligence

SN - 0162-8828

IS - 6

ER -