Discriminative shared Gaussian processes for multi-view and view-invariant facial expression recognition

Stefanos Eleftheriadis, Ognjen Rudovic, Maja Pantic

Research output: Contribution to journalArticleAcademicpeer-review

117 Citations (Scopus)

Abstract

Images of facial expressions are often captured from various views as a result of either head movements or variable camera position. Existing methods for multiview and/or view-invariant facial expression recognition typically perform classification of the observed expression using either classifiers learned separately for each view or a single classifier learned for all views. However, these approaches ignore the fact that different views of a facial expression are just different manifestations of the same facial expression. By accounting for this redundancy, we can design more effective classifiers for the target task. To this end, we propose a discriminative shared Gaussian process latent variable model (DS-GPLVM) for multiview and view-invariant classification of facial expressions from multiple views. In this model, we first learn a discriminative manifold shared by multiple views of a facial expression. Subsequently, we perform facial expression classification in the expression manifold. Finally, classification of an observed facial expression is carried out either in the view-invariant manner (using only a single view of the expression) or in the multiview manner (using multiple views of the expression). The proposed model can also be used to perform fusion of different facial features in a principled manner. We validate the proposed DS-GPLVM on both posed and spontaneously displayed facial expressions from three publicly available datasets (MultiPIE, labeled face parts in the wild, and static facial expressions in the wild). We show that this model outperforms the state-of-the-art methods for multiview and view-invariant facial expression classification, and several state-of-the-art methods for multiview learning and feature fusion.
Original languageUndefined
Pages (from-to)189-204
Number of pages16
JournalIEEE transactions on image processing
Volume24
Issue number1
DOIs
Publication statusPublished - Jan 2015

Keywords

  • EC Grant Agreement nr.: FP7/2007-2013
  • EWI-25804
  • HMI-HF: Human Factors
  • View-invariant
  • METIS-314908
  • EC Grant Agreement nr.: FP7/611153
  • IR-98117
  • Gaussian processes
  • Facial expression recognition
  • multi-view learning

Cite this

Eleftheriadis, Stefanos ; Rudovic, Ognjen ; Pantic, Maja. / Discriminative shared Gaussian processes for multi-view and view-invariant facial expression recognition. In: IEEE transactions on image processing. 2015 ; Vol. 24, No. 1. pp. 189-204.
@article{80d043fba98944659aa20cf0c286a4df,
title = "Discriminative shared Gaussian processes for multi-view and view-invariant facial expression recognition",
abstract = "Images of facial expressions are often captured from various views as a result of either head movements or variable camera position. Existing methods for multiview and/or view-invariant facial expression recognition typically perform classification of the observed expression using either classifiers learned separately for each view or a single classifier learned for all views. However, these approaches ignore the fact that different views of a facial expression are just different manifestations of the same facial expression. By accounting for this redundancy, we can design more effective classifiers for the target task. To this end, we propose a discriminative shared Gaussian process latent variable model (DS-GPLVM) for multiview and view-invariant classification of facial expressions from multiple views. In this model, we first learn a discriminative manifold shared by multiple views of a facial expression. Subsequently, we perform facial expression classification in the expression manifold. Finally, classification of an observed facial expression is carried out either in the view-invariant manner (using only a single view of the expression) or in the multiview manner (using multiple views of the expression). The proposed model can also be used to perform fusion of different facial features in a principled manner. We validate the proposed DS-GPLVM on both posed and spontaneously displayed facial expressions from three publicly available datasets (MultiPIE, labeled face parts in the wild, and static facial expressions in the wild). We show that this model outperforms the state-of-the-art methods for multiview and view-invariant facial expression classification, and several state-of-the-art methods for multiview learning and feature fusion.",
keywords = "EC Grant Agreement nr.: FP7/2007-2013, EWI-25804, HMI-HF: Human Factors, View-invariant, METIS-314908, EC Grant Agreement nr.: FP7/611153, IR-98117, Gaussian processes, Facial expression recognition, multi-view learning",
author = "Stefanos Eleftheriadis and Ognjen Rudovic and Maja Pantic",
note = "eemcs-eprint-25804",
year = "2015",
month = "1",
doi = "10.1109/TIP.2014.2375634",
language = "Undefined",
volume = "24",
pages = "189--204",
journal = "IEEE transactions on image processing",
issn = "1057-7149",
publisher = "IEEE",
number = "1",

}

Discriminative shared Gaussian processes for multi-view and view-invariant facial expression recognition. / Eleftheriadis, Stefanos; Rudovic, Ognjen; Pantic, Maja.

In: IEEE transactions on image processing, Vol. 24, No. 1, 01.2015, p. 189-204.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Discriminative shared Gaussian processes for multi-view and view-invariant facial expression recognition

AU - Eleftheriadis, Stefanos

AU - Rudovic, Ognjen

AU - Pantic, Maja

N1 - eemcs-eprint-25804

PY - 2015/1

Y1 - 2015/1

N2 - Images of facial expressions are often captured from various views as a result of either head movements or variable camera position. Existing methods for multiview and/or view-invariant facial expression recognition typically perform classification of the observed expression using either classifiers learned separately for each view or a single classifier learned for all views. However, these approaches ignore the fact that different views of a facial expression are just different manifestations of the same facial expression. By accounting for this redundancy, we can design more effective classifiers for the target task. To this end, we propose a discriminative shared Gaussian process latent variable model (DS-GPLVM) for multiview and view-invariant classification of facial expressions from multiple views. In this model, we first learn a discriminative manifold shared by multiple views of a facial expression. Subsequently, we perform facial expression classification in the expression manifold. Finally, classification of an observed facial expression is carried out either in the view-invariant manner (using only a single view of the expression) or in the multiview manner (using multiple views of the expression). The proposed model can also be used to perform fusion of different facial features in a principled manner. We validate the proposed DS-GPLVM on both posed and spontaneously displayed facial expressions from three publicly available datasets (MultiPIE, labeled face parts in the wild, and static facial expressions in the wild). We show that this model outperforms the state-of-the-art methods for multiview and view-invariant facial expression classification, and several state-of-the-art methods for multiview learning and feature fusion.

AB - Images of facial expressions are often captured from various views as a result of either head movements or variable camera position. Existing methods for multiview and/or view-invariant facial expression recognition typically perform classification of the observed expression using either classifiers learned separately for each view or a single classifier learned for all views. However, these approaches ignore the fact that different views of a facial expression are just different manifestations of the same facial expression. By accounting for this redundancy, we can design more effective classifiers for the target task. To this end, we propose a discriminative shared Gaussian process latent variable model (DS-GPLVM) for multiview and view-invariant classification of facial expressions from multiple views. In this model, we first learn a discriminative manifold shared by multiple views of a facial expression. Subsequently, we perform facial expression classification in the expression manifold. Finally, classification of an observed facial expression is carried out either in the view-invariant manner (using only a single view of the expression) or in the multiview manner (using multiple views of the expression). The proposed model can also be used to perform fusion of different facial features in a principled manner. We validate the proposed DS-GPLVM on both posed and spontaneously displayed facial expressions from three publicly available datasets (MultiPIE, labeled face parts in the wild, and static facial expressions in the wild). We show that this model outperforms the state-of-the-art methods for multiview and view-invariant facial expression classification, and several state-of-the-art methods for multiview learning and feature fusion.

KW - EC Grant Agreement nr.: FP7/2007-2013

KW - EWI-25804

KW - HMI-HF: Human Factors

KW - View-invariant

KW - METIS-314908

KW - EC Grant Agreement nr.: FP7/611153

KW - IR-98117

KW - Gaussian processes

KW - Facial expression recognition

KW - multi-view learning

U2 - 10.1109/TIP.2014.2375634

DO - 10.1109/TIP.2014.2375634

M3 - Article

VL - 24

SP - 189

EP - 204

JO - IEEE transactions on image processing

JF - IEEE transactions on image processing

SN - 1057-7149

IS - 1

ER -