Model-free 3D face shape reconstruction from video sequences

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

2 Downloads (Pure)

Abstract

In forensic comparison of facial video data, often only the best quality frontal face frames are selected, and hence much video data is ignored. To improve 2D facial comparison for law enforcement and forensic investigation, we introduce a model-free 3D shape reconstruction algorithm based on 2D landmarks. The algorithm uses around 20 landmarks on the face and combines the structure information of multiple frames. Model based 3D reconstruction methods, such as Morphable Models, reconstruct a 3D face shape model that is strongly biased towards the average face. Therefore, we don't use statistical face shape models in our model-free approach. The 3D landmark reconstruction algorithm simultaneously estimates the shape, pose and position of the face, based only on the fact that all images in the sequence are recorded using a single calibrated camera. The algorithm iteratively updates the reconstruction by including new frames, while maintaining the consistency of the reconstruction. We demonstrate the convergence properties of the method reflected in the 2D reprojection error and the 3D error with respect to a ground truth model. We show that the quality of the reconstruction depends on the noise on the landmarks. In a second experiment we show that the method can be used on realistic face shape data with a styrofoam head model.
Original languageEnglish
Title of host publicationProceedings of the 34rd Symposium on Information Theory in the Benelux and the 3rd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux
Place of PublicationEnschede
PublisherUniversity of Twente
Pages228-235
Number of pages8
ISBN (Print)978-90-365-0000-5
Publication statusPublished - May 2013
Event34th WIC Symposium on Information Theory in the Benelux 2013 - Leuven, Belgium
Duration: 30 May 201331 May 2013
Conference number: 34

Conference

Conference34th WIC Symposium on Information Theory in the Benelux 2013
CountryBelgium
CityLeuven
Period30/05/1331/05/13

Fingerprint

Law enforcement
Cameras
Experiments

Keywords

  • SCS-Safety
  • Landmark-based
  • Model-free
  • 3D face shape reconstruction

Cite this

van Dam, C., Veldhuis, R., & Spreeuwers, L. (2013). Model-free 3D face shape reconstruction from video sequences. In Proceedings of the 34rd Symposium on Information Theory in the Benelux and the 3rd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux (pp. 228-235). Enschede: University of Twente.
van Dam, Chris ; Veldhuis, Raymond ; Spreeuwers, Luuk. / Model-free 3D face shape reconstruction from video sequences. Proceedings of the 34rd Symposium on Information Theory in the Benelux and the 3rd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Enschede : University of Twente, 2013. pp. 228-235
@inproceedings{62f3f7e031a54f56abb0753b97dd00fd,
title = "Model-free 3D face shape reconstruction from video sequences",
abstract = "In forensic comparison of facial video data, often only the best quality frontal face frames are selected, and hence much video data is ignored. To improve 2D facial comparison for law enforcement and forensic investigation, we introduce a model-free 3D shape reconstruction algorithm based on 2D landmarks. The algorithm uses around 20 landmarks on the face and combines the structure information of multiple frames. Model based 3D reconstruction methods, such as Morphable Models, reconstruct a 3D face shape model that is strongly biased towards the average face. Therefore, we don't use statistical face shape models in our model-free approach. The 3D landmark reconstruction algorithm simultaneously estimates the shape, pose and position of the face, based only on the fact that all images in the sequence are recorded using a single calibrated camera. The algorithm iteratively updates the reconstruction by including new frames, while maintaining the consistency of the reconstruction. We demonstrate the convergence properties of the method reflected in the 2D reprojection error and the 3D error with respect to a ground truth model. We show that the quality of the reconstruction depends on the noise on the landmarks. In a second experiment we show that the method can be used on realistic face shape data with a styrofoam head model.",
keywords = "SCS-Safety, Landmark-based, Model-free, 3D face shape reconstruction",
author = "{van Dam}, Chris and Raymond Veldhuis and Luuk Spreeuwers",
note = "http://www.w-i-c.org/upload/files/proceedings_wic2013_final_authors.pdf",
year = "2013",
month = "5",
language = "English",
isbn = "978-90-365-0000-5",
pages = "228--235",
booktitle = "Proceedings of the 34rd Symposium on Information Theory in the Benelux and the 3rd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux",
publisher = "University of Twente",
address = "Netherlands",

}

van Dam, C, Veldhuis, R & Spreeuwers, L 2013, Model-free 3D face shape reconstruction from video sequences. in Proceedings of the 34rd Symposium on Information Theory in the Benelux and the 3rd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. University of Twente, Enschede, pp. 228-235, 34th WIC Symposium on Information Theory in the Benelux 2013, Leuven, Belgium, 30/05/13.

Model-free 3D face shape reconstruction from video sequences. / van Dam, Chris; Veldhuis, Raymond; Spreeuwers, Luuk.

Proceedings of the 34rd Symposium on Information Theory in the Benelux and the 3rd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Enschede : University of Twente, 2013. p. 228-235.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

TY - GEN

T1 - Model-free 3D face shape reconstruction from video sequences

AU - van Dam, Chris

AU - Veldhuis, Raymond

AU - Spreeuwers, Luuk

N1 - http://www.w-i-c.org/upload/files/proceedings_wic2013_final_authors.pdf

PY - 2013/5

Y1 - 2013/5

N2 - In forensic comparison of facial video data, often only the best quality frontal face frames are selected, and hence much video data is ignored. To improve 2D facial comparison for law enforcement and forensic investigation, we introduce a model-free 3D shape reconstruction algorithm based on 2D landmarks. The algorithm uses around 20 landmarks on the face and combines the structure information of multiple frames. Model based 3D reconstruction methods, such as Morphable Models, reconstruct a 3D face shape model that is strongly biased towards the average face. Therefore, we don't use statistical face shape models in our model-free approach. The 3D landmark reconstruction algorithm simultaneously estimates the shape, pose and position of the face, based only on the fact that all images in the sequence are recorded using a single calibrated camera. The algorithm iteratively updates the reconstruction by including new frames, while maintaining the consistency of the reconstruction. We demonstrate the convergence properties of the method reflected in the 2D reprojection error and the 3D error with respect to a ground truth model. We show that the quality of the reconstruction depends on the noise on the landmarks. In a second experiment we show that the method can be used on realistic face shape data with a styrofoam head model.

AB - In forensic comparison of facial video data, often only the best quality frontal face frames are selected, and hence much video data is ignored. To improve 2D facial comparison for law enforcement and forensic investigation, we introduce a model-free 3D shape reconstruction algorithm based on 2D landmarks. The algorithm uses around 20 landmarks on the face and combines the structure information of multiple frames. Model based 3D reconstruction methods, such as Morphable Models, reconstruct a 3D face shape model that is strongly biased towards the average face. Therefore, we don't use statistical face shape models in our model-free approach. The 3D landmark reconstruction algorithm simultaneously estimates the shape, pose and position of the face, based only on the fact that all images in the sequence are recorded using a single calibrated camera. The algorithm iteratively updates the reconstruction by including new frames, while maintaining the consistency of the reconstruction. We demonstrate the convergence properties of the method reflected in the 2D reprojection error and the 3D error with respect to a ground truth model. We show that the quality of the reconstruction depends on the noise on the landmarks. In a second experiment we show that the method can be used on realistic face shape data with a styrofoam head model.

KW - SCS-Safety

KW - Landmark-based

KW - Model-free

KW - 3D face shape reconstruction

M3 - Conference contribution

SN - 978-90-365-0000-5

SP - 228

EP - 235

BT - Proceedings of the 34rd Symposium on Information Theory in the Benelux and the 3rd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux

PB - University of Twente

CY - Enschede

ER -

van Dam C, Veldhuis R, Spreeuwers L. Model-free 3D face shape reconstruction from video sequences. In Proceedings of the 34rd Symposium on Information Theory in the Benelux and the 3rd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Enschede: University of Twente. 2013. p. 228-235