Using 3D Morphable Models for face recognition in video

R.T.A. van Rootseler, L.J. Spreeuwers, R.N.J. Veldhuis

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

53 Downloads (Pure)

Abstract

The 3D Morphable Face Model (3DMM) has been used for over a decade for creating 3D models from single images of faces. This model is based on a PCA model of the 3D shape and texture generated from a limited number of 3D scans. The goal of fitting a 3DMM to an image is to find the model coefficients, the lighting and other imaging variables from which we can remodel that image as accurately as possible. The model coefficients consist of texture and of shape descriptors, and can without further processing be used in verification and recognition experiments. Until now little research has been performed into the influence of the diverse parameters of the 3DMM on the recognition performance. In this paper we will introduce a Bayesian-based method for texture backmapping from multiple images. Using the information from multiple (non-frontal) views we construct a frontal view which can be used as input to 2D face recognition software. We also show how the number of triangles used in the fitting proces influences the recognition performance using the shape descriptors. The verification results of the 3DMM are compared to state-of-the-art 2D face recognition software on the MultiPIE dataset. The 2D FR software outperforms the Morphable Model, but the Morphable Model can be useful as a preprocesser to synthesize a frontal view from a non-frontal view and also combine images with multiple views to a single frontal view. We show results for this preprocessing technique by using an average face shape, a fitted face shape, with a MM texture, with the original texture and with a hybrid texture. The preprocessor has improved the verification results significantly on the dataset.
Original languageEnglish
Title of host publicationProceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux
Place of PublicationEnschede
PublisherWerkgemeenschap voor Informatie- en Communicatietheorie (WIC)
Pages235-242
Number of pages8
ISBN (Print)978-90-365-3383-6
Publication statusPublished - May 2012
Event33rd WIC Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux 2012 - Boekelo, Netherlands
Duration: 24 May 201225 May 2012
Conference number: 33

Conference

Conference33rd WIC Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux 2012
CountryNetherlands
CityBoekelo
Period24/05/1225/05/12

Fingerprint

Face recognition
Textures
Lighting
Imaging techniques

Keywords

  • METIS-286370
  • IR-80462
  • Video processing
  • SCS-Safety
  • Image Processing
  • Biometrics
  • EWI-21886
  • Morphable Models

Cite this

van Rootseler, R. T. A., Spreeuwers, L. J., & Veldhuis, R. N. J. (2012). Using 3D Morphable Models for face recognition in video. In Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux (pp. 235-242). Enschede: Werkgemeenschap voor Informatie- en Communicatietheorie (WIC).
van Rootseler, R.T.A. ; Spreeuwers, L.J. ; Veldhuis, R.N.J. / Using 3D Morphable Models for face recognition in video. Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Enschede : Werkgemeenschap voor Informatie- en Communicatietheorie (WIC), 2012. pp. 235-242
@inproceedings{9c579a70def24aeb98696a38858a1333,
title = "Using 3D Morphable Models for face recognition in video",
abstract = "The 3D Morphable Face Model (3DMM) has been used for over a decade for creating 3D models from single images of faces. This model is based on a PCA model of the 3D shape and texture generated from a limited number of 3D scans. The goal of fitting a 3DMM to an image is to find the model coefficients, the lighting and other imaging variables from which we can remodel that image as accurately as possible. The model coefficients consist of texture and of shape descriptors, and can without further processing be used in verification and recognition experiments. Until now little research has been performed into the influence of the diverse parameters of the 3DMM on the recognition performance. In this paper we will introduce a Bayesian-based method for texture backmapping from multiple images. Using the information from multiple (non-frontal) views we construct a frontal view which can be used as input to 2D face recognition software. We also show how the number of triangles used in the fitting proces influences the recognition performance using the shape descriptors. The verification results of the 3DMM are compared to state-of-the-art 2D face recognition software on the MultiPIE dataset. The 2D FR software outperforms the Morphable Model, but the Morphable Model can be useful as a preprocesser to synthesize a frontal view from a non-frontal view and also combine images with multiple views to a single frontal view. We show results for this preprocessing technique by using an average face shape, a fitted face shape, with a MM texture, with the original texture and with a hybrid texture. The preprocessor has improved the verification results significantly on the dataset.",
keywords = "METIS-286370, IR-80462, Video processing, SCS-Safety, Image Processing, Biometrics, EWI-21886, Morphable Models",
author = "{van Rootseler}, R.T.A. and L.J. Spreeuwers and R.N.J. Veldhuis",
year = "2012",
month = "5",
language = "English",
isbn = "978-90-365-3383-6",
pages = "235--242",
booktitle = "Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux",
publisher = "Werkgemeenschap voor Informatie- en Communicatietheorie (WIC)",
address = "Netherlands",

}

van Rootseler, RTA, Spreeuwers, LJ & Veldhuis, RNJ 2012, Using 3D Morphable Models for face recognition in video. in Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Werkgemeenschap voor Informatie- en Communicatietheorie (WIC), Enschede, pp. 235-242, 33rd WIC Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux 2012, Boekelo, Netherlands, 24/05/12.

Using 3D Morphable Models for face recognition in video. / van Rootseler, R.T.A.; Spreeuwers, L.J.; Veldhuis, R.N.J.

Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Enschede : Werkgemeenschap voor Informatie- en Communicatietheorie (WIC), 2012. p. 235-242.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

TY - GEN

T1 - Using 3D Morphable Models for face recognition in video

AU - van Rootseler, R.T.A.

AU - Spreeuwers, L.J.

AU - Veldhuis, R.N.J.

PY - 2012/5

Y1 - 2012/5

N2 - The 3D Morphable Face Model (3DMM) has been used for over a decade for creating 3D models from single images of faces. This model is based on a PCA model of the 3D shape and texture generated from a limited number of 3D scans. The goal of fitting a 3DMM to an image is to find the model coefficients, the lighting and other imaging variables from which we can remodel that image as accurately as possible. The model coefficients consist of texture and of shape descriptors, and can without further processing be used in verification and recognition experiments. Until now little research has been performed into the influence of the diverse parameters of the 3DMM on the recognition performance. In this paper we will introduce a Bayesian-based method for texture backmapping from multiple images. Using the information from multiple (non-frontal) views we construct a frontal view which can be used as input to 2D face recognition software. We also show how the number of triangles used in the fitting proces influences the recognition performance using the shape descriptors. The verification results of the 3DMM are compared to state-of-the-art 2D face recognition software on the MultiPIE dataset. The 2D FR software outperforms the Morphable Model, but the Morphable Model can be useful as a preprocesser to synthesize a frontal view from a non-frontal view and also combine images with multiple views to a single frontal view. We show results for this preprocessing technique by using an average face shape, a fitted face shape, with a MM texture, with the original texture and with a hybrid texture. The preprocessor has improved the verification results significantly on the dataset.

AB - The 3D Morphable Face Model (3DMM) has been used for over a decade for creating 3D models from single images of faces. This model is based on a PCA model of the 3D shape and texture generated from a limited number of 3D scans. The goal of fitting a 3DMM to an image is to find the model coefficients, the lighting and other imaging variables from which we can remodel that image as accurately as possible. The model coefficients consist of texture and of shape descriptors, and can without further processing be used in verification and recognition experiments. Until now little research has been performed into the influence of the diverse parameters of the 3DMM on the recognition performance. In this paper we will introduce a Bayesian-based method for texture backmapping from multiple images. Using the information from multiple (non-frontal) views we construct a frontal view which can be used as input to 2D face recognition software. We also show how the number of triangles used in the fitting proces influences the recognition performance using the shape descriptors. The verification results of the 3DMM are compared to state-of-the-art 2D face recognition software on the MultiPIE dataset. The 2D FR software outperforms the Morphable Model, but the Morphable Model can be useful as a preprocesser to synthesize a frontal view from a non-frontal view and also combine images with multiple views to a single frontal view. We show results for this preprocessing technique by using an average face shape, a fitted face shape, with a MM texture, with the original texture and with a hybrid texture. The preprocessor has improved the verification results significantly on the dataset.

KW - METIS-286370

KW - IR-80462

KW - Video processing

KW - SCS-Safety

KW - Image Processing

KW - Biometrics

KW - EWI-21886

KW - Morphable Models

M3 - Conference contribution

SN - 978-90-365-3383-6

SP - 235

EP - 242

BT - Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux

PB - Werkgemeenschap voor Informatie- en Communicatietheorie (WIC)

CY - Enschede

ER -

van Rootseler RTA, Spreeuwers LJ, Veldhuis RNJ. Using 3D Morphable Models for face recognition in video. In Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Enschede: Werkgemeenschap voor Informatie- en Communicatietheorie (WIC). 2012. p. 235-242