Facial Expression Invariant Head Pose Normalization using Gaussian Process Regression

Ognjen Rudovic, Ioannis Patras, Maja Pantic

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

8 Citations (Scopus)
11 Downloads (Pure)

Abstract

We present a regression-based scheme for facialexpression-invariant head pose normalization. We address the problem by mapping the locations of 2D facial points (e.g. mouth corners) from non-frontal poses to the frontal pose. This is done in two steps. First, we propose a head pose estimator that maps the input 2D facial point locations into a head-pose space defined by a low dimensional manifold attained by means of multi-class LDA. Then, to learn the mappings between a discrete set of non-frontal head poses and the frontal pose, we propose using a Gaussian Process Regression (GPR) model for each pair of target poses (i.e. a non-frontal and the frontal pose). During testing, the head pose estimator is used to activate the most relevant GPR model which is later applied to project the locations of 2D facial landmarks from an arbitrary pose (that does not have to be one of the training poses) to the frontal pose. In our experiments we show that the proposed scheme (i) performs accurately for continuous head pose in the range from 0° to 45° pan rotation and from 0° to 30° tilt rotation despite the fact that the training was conducted only on a set of discrete poses, (ii) handles successfully both expressive and expressionless faces (even in cases when some of the expression categories were missing in certain poses during the training), and (iii) outperforms both 3D Point Distribution Model (3D-PDM) and Linear Regression (LR) model that are used as baseline methods for pose normalization. The proposed method is experimentally evaluated on data from the BU蚠3DFE facial expression database.
Original languageUndefined
Title of host publicationIEEE Conference on Computer Vision and Pattern Recognition (CVPR '10), Workshop CVPR for Human Communicative Behaviour Analysis (CVPR4HB)
Place of PublicationUSA
PublisherIEEE Computer Society
Pages28-33
Number of pages6
ISBN (Print)978-1-4244-7029-7
DOIs
Publication statusPublished - 18 Jun 2010

Publication series

Name
PublisherIEEE Computer Society
Volume3

Keywords

  • IR-75976
  • METIS-276363
  • EC Grant Agreement nr.: FP7/211486
  • HMI-MI: MULTIMODAL INTERACTIONS
  • EWI-19550

Cite this

Rudovic, O., Patras, I., & Pantic, M. (2010). Facial Expression Invariant Head Pose Normalization using Gaussian Process Regression. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR '10), Workshop CVPR for Human Communicative Behaviour Analysis (CVPR4HB) (pp. 28-33). USA: IEEE Computer Society. https://doi.org/10.1109/CVPRW.2010.5543269
Rudovic, Ognjen ; Patras, Ioannis ; Pantic, Maja. / Facial Expression Invariant Head Pose Normalization using Gaussian Process Regression. IEEE Conference on Computer Vision and Pattern Recognition (CVPR '10), Workshop CVPR for Human Communicative Behaviour Analysis (CVPR4HB). USA : IEEE Computer Society, 2010. pp. 28-33
@inproceedings{f2c70a1daba94ae089a23bc18a68a4d5,
title = "Facial Expression Invariant Head Pose Normalization using Gaussian Process Regression",
abstract = "We present a regression-based scheme for facialexpression-invariant head pose normalization. We address the problem by mapping the locations of 2D facial points (e.g. mouth corners) from non-frontal poses to the frontal pose. This is done in two steps. First, we propose a head pose estimator that maps the input 2D facial point locations into a head-pose space defined by a low dimensional manifold attained by means of multi-class LDA. Then, to learn the mappings between a discrete set of non-frontal head poses and the frontal pose, we propose using a Gaussian Process Regression (GPR) model for each pair of target poses (i.e. a non-frontal and the frontal pose). During testing, the head pose estimator is used to activate the most relevant GPR model which is later applied to project the locations of 2D facial landmarks from an arbitrary pose (that does not have to be one of the training poses) to the frontal pose. In our experiments we show that the proposed scheme (i) performs accurately for continuous head pose in the range from 0° to 45° pan rotation and from 0° to 30° tilt rotation despite the fact that the training was conducted only on a set of discrete poses, (ii) handles successfully both expressive and expressionless faces (even in cases when some of the expression categories were missing in certain poses during the training), and (iii) outperforms both 3D Point Distribution Model (3D-PDM) and Linear Regression (LR) model that are used as baseline methods for pose normalization. The proposed method is experimentally evaluated on data from the BU蚠3DFE facial expression database.",
keywords = "IR-75976, METIS-276363, EC Grant Agreement nr.: FP7/211486, HMI-MI: MULTIMODAL INTERACTIONS, EWI-19550",
author = "Ognjen Rudovic and Ioannis Patras and Maja Pantic",
year = "2010",
month = "6",
day = "18",
doi = "10.1109/CVPRW.2010.5543269",
language = "Undefined",
isbn = "978-1-4244-7029-7",
publisher = "IEEE Computer Society",
pages = "28--33",
booktitle = "IEEE Conference on Computer Vision and Pattern Recognition (CVPR '10), Workshop CVPR for Human Communicative Behaviour Analysis (CVPR4HB)",
address = "United States",

}

Rudovic, O, Patras, I & Pantic, M 2010, Facial Expression Invariant Head Pose Normalization using Gaussian Process Regression. in IEEE Conference on Computer Vision and Pattern Recognition (CVPR '10), Workshop CVPR for Human Communicative Behaviour Analysis (CVPR4HB). IEEE Computer Society, USA, pp. 28-33. https://doi.org/10.1109/CVPRW.2010.5543269

Facial Expression Invariant Head Pose Normalization using Gaussian Process Regression. / Rudovic, Ognjen; Patras, Ioannis; Pantic, Maja.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR '10), Workshop CVPR for Human Communicative Behaviour Analysis (CVPR4HB). USA : IEEE Computer Society, 2010. p. 28-33.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Facial Expression Invariant Head Pose Normalization using Gaussian Process Regression

AU - Rudovic, Ognjen

AU - Patras, Ioannis

AU - Pantic, Maja

PY - 2010/6/18

Y1 - 2010/6/18

N2 - We present a regression-based scheme for facialexpression-invariant head pose normalization. We address the problem by mapping the locations of 2D facial points (e.g. mouth corners) from non-frontal poses to the frontal pose. This is done in two steps. First, we propose a head pose estimator that maps the input 2D facial point locations into a head-pose space defined by a low dimensional manifold attained by means of multi-class LDA. Then, to learn the mappings between a discrete set of non-frontal head poses and the frontal pose, we propose using a Gaussian Process Regression (GPR) model for each pair of target poses (i.e. a non-frontal and the frontal pose). During testing, the head pose estimator is used to activate the most relevant GPR model which is later applied to project the locations of 2D facial landmarks from an arbitrary pose (that does not have to be one of the training poses) to the frontal pose. In our experiments we show that the proposed scheme (i) performs accurately for continuous head pose in the range from 0° to 45° pan rotation and from 0° to 30° tilt rotation despite the fact that the training was conducted only on a set of discrete poses, (ii) handles successfully both expressive and expressionless faces (even in cases when some of the expression categories were missing in certain poses during the training), and (iii) outperforms both 3D Point Distribution Model (3D-PDM) and Linear Regression (LR) model that are used as baseline methods for pose normalization. The proposed method is experimentally evaluated on data from the BU蚠3DFE facial expression database.

AB - We present a regression-based scheme for facialexpression-invariant head pose normalization. We address the problem by mapping the locations of 2D facial points (e.g. mouth corners) from non-frontal poses to the frontal pose. This is done in two steps. First, we propose a head pose estimator that maps the input 2D facial point locations into a head-pose space defined by a low dimensional manifold attained by means of multi-class LDA. Then, to learn the mappings between a discrete set of non-frontal head poses and the frontal pose, we propose using a Gaussian Process Regression (GPR) model for each pair of target poses (i.e. a non-frontal and the frontal pose). During testing, the head pose estimator is used to activate the most relevant GPR model which is later applied to project the locations of 2D facial landmarks from an arbitrary pose (that does not have to be one of the training poses) to the frontal pose. In our experiments we show that the proposed scheme (i) performs accurately for continuous head pose in the range from 0° to 45° pan rotation and from 0° to 30° tilt rotation despite the fact that the training was conducted only on a set of discrete poses, (ii) handles successfully both expressive and expressionless faces (even in cases when some of the expression categories were missing in certain poses during the training), and (iii) outperforms both 3D Point Distribution Model (3D-PDM) and Linear Regression (LR) model that are used as baseline methods for pose normalization. The proposed method is experimentally evaluated on data from the BU蚠3DFE facial expression database.

KW - IR-75976

KW - METIS-276363

KW - EC Grant Agreement nr.: FP7/211486

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - EWI-19550

U2 - 10.1109/CVPRW.2010.5543269

DO - 10.1109/CVPRW.2010.5543269

M3 - Conference contribution

SN - 978-1-4244-7029-7

SP - 28

EP - 33

BT - IEEE Conference on Computer Vision and Pattern Recognition (CVPR '10), Workshop CVPR for Human Communicative Behaviour Analysis (CVPR4HB)

PB - IEEE Computer Society

CY - USA

ER -

Rudovic O, Patras I, Pantic M. Facial Expression Invariant Head Pose Normalization using Gaussian Process Regression. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR '10), Workshop CVPR for Human Communicative Behaviour Analysis (CVPR4HB). USA: IEEE Computer Society. 2010. p. 28-33 https://doi.org/10.1109/CVPRW.2010.5543269