A dynamic approach to the recognition of 3D facial expressions and their temporal models

Georgia Sandbach; Stefanos Zafeiriou; Maja Pantic; Daniel Rueckert

doi:10.1109/FG.2011.5771434

A dynamic approach to the recognition of 3D facial expressions and their temporal models

Georgia Sandbach, Stefanos Zafeiriou, Maja Pantic, Daniel Rueckert

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

70 Citations (Scopus)

12 Downloads (Pure)

Abstract

In this paper we propose a method that exploits 3D motion-based features between frames of 3D facial geometry sequences for dynamic facial expression recognition. An expressive sequence is modeled to contain an onset followed by an apex and an offset. Feature selection methods are applied in order to extract features for each of the onset and offset segments of the expression. These features are then used to train a Hidden Markov Model in order to model the full temporal dynamics of the expression. The proposed fully automatic system was tested in a subset of the BU-4DFE database for the recognition of happiness, anger and surprise. Comparisons with a similar system based on the motion extracted from facial intensity images was also performed. The attained results suggest that the use of the 3D information does indeed improve the recognition accuracy when compared to the 2D data.

Original language	Undefined
Title of host publication	IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011)
Place of Publication	USA
Publisher	IEEE
Pages	406-413
Number of pages	8
ISBN (Print)	978-1-4244-9140-7
DOIs	https://doi.org/10.1109/FG.2011.5771434
Publication status	Published - Mar 2011
Event	9th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2011 - Santa Barbara, United States Duration: 21 Mar 2011 → 25 Mar 2011 Conference number: 9

Publication series

Name
Publisher	IEEE Computer Society

Conference

Conference	9th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2011
Abbreviated title	FG
Country/Territory	United States
City	Santa Barbara
Period	21/03/11 → 25/03/11

Keywords

METIS-285046
IR-79508
Face Recognition
Image segmentation
Image sequences
HMI-MI: MULTIMODAL INTERACTIONS
Hidden Markov models
Three dimensional displays
Training
EC Grant Agreement nr.: FP7/231287
EWI-21357
Feature extraction

Access to Document

10.1109/FG.2011.5771434

http://eprints.eemcs.utwente.nl/secure2/21357/01/pantic_a_dynamic_approach_to_the_recognition_of_3D_facial_expressions_and_their.pdf

Cite this

@inproceedings{c69e1b5aaefc4157abc025de4782e253,

title = "A dynamic approach to the recognition of 3D facial expressions and their temporal models",

abstract = "In this paper we propose a method that exploits 3D motion-based features between frames of 3D facial geometry sequences for dynamic facial expression recognition. An expressive sequence is modeled to contain an onset followed by an apex and an offset. Feature selection methods are applied in order to extract features for each of the onset and offset segments of the expression. These features are then used to train a Hidden Markov Model in order to model the full temporal dynamics of the expression. The proposed fully automatic system was tested in a subset of the BU-4DFE database for the recognition of happiness, anger and surprise. Comparisons with a similar system based on the motion extracted from facial intensity images was also performed. The attained results suggest that the use of the 3D information does indeed improve the recognition accuracy when compared to the 2D data.",

keywords = "METIS-285046, IR-79508, Face Recognition, Image segmentation, Image sequences, HMI-MI: MULTIMODAL INTERACTIONS, Hidden Markov models, Three dimensional displays, Training, EC Grant Agreement nr.: FP7/231287, EWI-21357, Feature extraction",

author = "Georgia Sandbach and Stefanos Zafeiriou and Maja Pantic and Daniel Rueckert",

note = "eemcs-eprint-21357 ; 9th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2011 ; Conference date: 21-03-2011 Through 25-03-2011",

year = "2011",

month = mar,

doi = "10.1109/FG.2011.5771434",

language = "Undefined",

isbn = "978-1-4244-9140-7",

publisher = "IEEE",

pages = "406--413",

booktitle = "IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011)",

address = "United States",

}

Sandbach, G, Zafeiriou, S, Pantic, M & Rueckert, D 2011, A dynamic approach to the recognition of 3D facial expressions and their temporal models. in IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011). IEEE, USA, pp. 406-413, 9th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2011, Santa Barbara, California, United States, 21/03/11. https://doi.org/10.1109/FG.2011.5771434

A dynamic approach to the recognition of 3D facial expressions and their temporal models. / Sandbach, Georgia; Zafeiriou, Stefanos; Pantic, Maja et al.
IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011). USA: IEEE, 2011. p. 406-413.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - A dynamic approach to the recognition of 3D facial expressions and their temporal models

AU - Sandbach, Georgia

AU - Zafeiriou, Stefanos

AU - Pantic, Maja

AU - Rueckert, Daniel

N1 - Conference code: 9

PY - 2011/3

Y1 - 2011/3

N2 - In this paper we propose a method that exploits 3D motion-based features between frames of 3D facial geometry sequences for dynamic facial expression recognition. An expressive sequence is modeled to contain an onset followed by an apex and an offset. Feature selection methods are applied in order to extract features for each of the onset and offset segments of the expression. These features are then used to train a Hidden Markov Model in order to model the full temporal dynamics of the expression. The proposed fully automatic system was tested in a subset of the BU-4DFE database for the recognition of happiness, anger and surprise. Comparisons with a similar system based on the motion extracted from facial intensity images was also performed. The attained results suggest that the use of the 3D information does indeed improve the recognition accuracy when compared to the 2D data.

AB - In this paper we propose a method that exploits 3D motion-based features between frames of 3D facial geometry sequences for dynamic facial expression recognition. An expressive sequence is modeled to contain an onset followed by an apex and an offset. Feature selection methods are applied in order to extract features for each of the onset and offset segments of the expression. These features are then used to train a Hidden Markov Model in order to model the full temporal dynamics of the expression. The proposed fully automatic system was tested in a subset of the BU-4DFE database for the recognition of happiness, anger and surprise. Comparisons with a similar system based on the motion extracted from facial intensity images was also performed. The attained results suggest that the use of the 3D information does indeed improve the recognition accuracy when compared to the 2D data.

KW - METIS-285046

KW - IR-79508

KW - Face Recognition

KW - Image segmentation

KW - Image sequences

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - Hidden Markov models

KW - Three dimensional displays

KW - Training

KW - EC Grant Agreement nr.: FP7/231287

KW - EWI-21357

KW - Feature extraction

U2 - 10.1109/FG.2011.5771434

DO - 10.1109/FG.2011.5771434

M3 - Conference contribution

SN - 978-1-4244-9140-7

SP - 406

EP - 413

BT - IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011)

PB - IEEE

CY - USA

T2 - 9th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2011

Y2 - 21 March 2011 through 25 March 2011

ER -