Landmark-based model-free 3D face shape reconstruction from video sequences

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

2 Citations (Scopus)
49 Downloads (Pure)

Abstract

In forensic comparison of facial video data, often only the best quality frontal face frames are selected, and hence potentially useful video data is ignored. To improve 2D facial comparison for law enforcement and forensic investigation, we introduce a model-free 3D shape reconstruction algorithm based on 2D landmarks. The algorithm uses around 20 landmarks on the face and combines the structure information of multiple frames. Model based 3D reconstruction methods, such as Morphable Models, reconstruct a 3D face shape model that is strongly biased towards the average face. Therefore, we don't use statistical face shape models in our model-free approach. The 3D landmark reconstruction algorithm simultaneously estimates the shape, pose and position of the face, based only on the fact that all images in the sequence are recorded using a single calibrated camera. The algorithm iteratively updates the reconstruction by including new frames, while maintaining the consistency of the reconstruction. We demonstrate the convergence properties of the method reflected in the 2D reprojection error and the 3D error with respect to a ground truth model. We show that the quality of the reconstruction depends on the level of noise in the landmarks. In follow-up experiments we show that our method is able to reconstruct the 3D structure of a face, using a styrofoam head and real video data. The results of the real face data show the same behavior as the results of the simulated data, which indicates that our method is capable of reconstructing real facial structures, depending on the noise of the landmarks.
Original languageEnglish
Title of host publication2013 International Conference of the BIOSIG Special Interest Group (BIOSIG)
EditorsA. Broemme, C. Busch
PublisherGesellschaft für Informatik
Pages265-272
Number of pages5
ISBN (Print)978-3-88579-606-0
Publication statusPublished - 5 Sep 2013
Event12th International Conference of the Biometrics Special Interest Group, BIOSIG 2013 - Darmstadt, Germany
Duration: 4 Sep 20136 Sep 2013
Conference number: 12

Publication series

NameLecture Notes in Informatics (LNI) - Proceedings
PublisherGesellschaft fuer Informatik
VolumeP-212
ISSN (Print)1617-5468

Conference

Conference12th International Conference of the Biometrics Special Interest Group, BIOSIG 2013
Abbreviated titleBIOSIG 2013
CountryGermany
CityDarmstadt
Period4/09/136/09/13

Fingerprint

Law enforcement
Cameras
Experiments

Keywords

  • SCS-Safety
  • Landmarks
  • Face Recognition
  • 3D modelling

Cite this

van Dam, C., Veldhuis, R. N. J., & Spreeuwers, L. J. (2013). Landmark-based model-free 3D face shape reconstruction from video sequences. In A. Broemme, & C. Busch (Eds.), 2013 International Conference of the BIOSIG Special Interest Group (BIOSIG) (pp. 265-272). (Lecture Notes in Informatics (LNI) - Proceedings; Vol. P-212). Gesellschaft für Informatik.
van Dam, C. ; Veldhuis, Raymond N.J. ; Spreeuwers, Lieuwe Jan. / Landmark-based model-free 3D face shape reconstruction from video sequences. 2013 International Conference of the BIOSIG Special Interest Group (BIOSIG). editor / A. Broemme ; C. Busch. Gesellschaft für Informatik, 2013. pp. 265-272 (Lecture Notes in Informatics (LNI) - Proceedings).
@inproceedings{9fb8302d86d84c6db84180ce9869b428,
title = "Landmark-based model-free 3D face shape reconstruction from video sequences",
abstract = "In forensic comparison of facial video data, often only the best quality frontal face frames are selected, and hence potentially useful video data is ignored. To improve 2D facial comparison for law enforcement and forensic investigation, we introduce a model-free 3D shape reconstruction algorithm based on 2D landmarks. The algorithm uses around 20 landmarks on the face and combines the structure information of multiple frames. Model based 3D reconstruction methods, such as Morphable Models, reconstruct a 3D face shape model that is strongly biased towards the average face. Therefore, we don't use statistical face shape models in our model-free approach. The 3D landmark reconstruction algorithm simultaneously estimates the shape, pose and position of the face, based only on the fact that all images in the sequence are recorded using a single calibrated camera. The algorithm iteratively updates the reconstruction by including new frames, while maintaining the consistency of the reconstruction. We demonstrate the convergence properties of the method reflected in the 2D reprojection error and the 3D error with respect to a ground truth model. We show that the quality of the reconstruction depends on the level of noise in the landmarks. In follow-up experiments we show that our method is able to reconstruct the 3D structure of a face, using a styrofoam head and real video data. The results of the real face data show the same behavior as the results of the simulated data, which indicates that our method is capable of reconstructing real facial structures, depending on the noise of the landmarks.",
keywords = "SCS-Safety, Landmarks, Face Recognition, 3D modelling",
author = "{van Dam}, C. and Veldhuis, {Raymond N.J.} and Spreeuwers, {Lieuwe Jan}",
year = "2013",
month = "9",
day = "5",
language = "English",
isbn = "978-3-88579-606-0",
series = "Lecture Notes in Informatics (LNI) - Proceedings",
publisher = "Gesellschaft f{\"u}r Informatik",
pages = "265--272",
editor = "A. Broemme and C. Busch",
booktitle = "2013 International Conference of the BIOSIG Special Interest Group (BIOSIG)",
address = "Germany",

}

van Dam, C, Veldhuis, RNJ & Spreeuwers, LJ 2013, Landmark-based model-free 3D face shape reconstruction from video sequences. in A Broemme & C Busch (eds), 2013 International Conference of the BIOSIG Special Interest Group (BIOSIG). Lecture Notes in Informatics (LNI) - Proceedings, vol. P-212, Gesellschaft für Informatik, pp. 265-272, 12th International Conference of the Biometrics Special Interest Group, BIOSIG 2013, Darmstadt, Germany, 4/09/13.

Landmark-based model-free 3D face shape reconstruction from video sequences. / van Dam, C.; Veldhuis, Raymond N.J.; Spreeuwers, Lieuwe Jan.

2013 International Conference of the BIOSIG Special Interest Group (BIOSIG). ed. / A. Broemme; C. Busch. Gesellschaft für Informatik, 2013. p. 265-272 (Lecture Notes in Informatics (LNI) - Proceedings; Vol. P-212).

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Landmark-based model-free 3D face shape reconstruction from video sequences

AU - van Dam, C.

AU - Veldhuis, Raymond N.J.

AU - Spreeuwers, Lieuwe Jan

PY - 2013/9/5

Y1 - 2013/9/5

N2 - In forensic comparison of facial video data, often only the best quality frontal face frames are selected, and hence potentially useful video data is ignored. To improve 2D facial comparison for law enforcement and forensic investigation, we introduce a model-free 3D shape reconstruction algorithm based on 2D landmarks. The algorithm uses around 20 landmarks on the face and combines the structure information of multiple frames. Model based 3D reconstruction methods, such as Morphable Models, reconstruct a 3D face shape model that is strongly biased towards the average face. Therefore, we don't use statistical face shape models in our model-free approach. The 3D landmark reconstruction algorithm simultaneously estimates the shape, pose and position of the face, based only on the fact that all images in the sequence are recorded using a single calibrated camera. The algorithm iteratively updates the reconstruction by including new frames, while maintaining the consistency of the reconstruction. We demonstrate the convergence properties of the method reflected in the 2D reprojection error and the 3D error with respect to a ground truth model. We show that the quality of the reconstruction depends on the level of noise in the landmarks. In follow-up experiments we show that our method is able to reconstruct the 3D structure of a face, using a styrofoam head and real video data. The results of the real face data show the same behavior as the results of the simulated data, which indicates that our method is capable of reconstructing real facial structures, depending on the noise of the landmarks.

AB - In forensic comparison of facial video data, often only the best quality frontal face frames are selected, and hence potentially useful video data is ignored. To improve 2D facial comparison for law enforcement and forensic investigation, we introduce a model-free 3D shape reconstruction algorithm based on 2D landmarks. The algorithm uses around 20 landmarks on the face and combines the structure information of multiple frames. Model based 3D reconstruction methods, such as Morphable Models, reconstruct a 3D face shape model that is strongly biased towards the average face. Therefore, we don't use statistical face shape models in our model-free approach. The 3D landmark reconstruction algorithm simultaneously estimates the shape, pose and position of the face, based only on the fact that all images in the sequence are recorded using a single calibrated camera. The algorithm iteratively updates the reconstruction by including new frames, while maintaining the consistency of the reconstruction. We demonstrate the convergence properties of the method reflected in the 2D reprojection error and the 3D error with respect to a ground truth model. We show that the quality of the reconstruction depends on the level of noise in the landmarks. In follow-up experiments we show that our method is able to reconstruct the 3D structure of a face, using a styrofoam head and real video data. The results of the real face data show the same behavior as the results of the simulated data, which indicates that our method is capable of reconstructing real facial structures, depending on the noise of the landmarks.

KW - SCS-Safety

KW - Landmarks

KW - Face Recognition

KW - 3D modelling

M3 - Conference contribution

SN - 978-3-88579-606-0

T3 - Lecture Notes in Informatics (LNI) - Proceedings

SP - 265

EP - 272

BT - 2013 International Conference of the BIOSIG Special Interest Group (BIOSIG)

A2 - Broemme, A.

A2 - Busch, C.

PB - Gesellschaft für Informatik

ER -

van Dam C, Veldhuis RNJ, Spreeuwers LJ. Landmark-based model-free 3D face shape reconstruction from video sequences. In Broemme A, Busch C, editors, 2013 International Conference of the BIOSIG Special Interest Group (BIOSIG). Gesellschaft für Informatik. 2013. p. 265-272. (Lecture Notes in Informatics (LNI) - Proceedings).