From image sequence to frontal image: reconstruction of the unknown face: a forensic case

Christiaan van Dam

Research output: ThesisPhD Thesis - Research UT, graduation UTAcademic

40 Downloads (Pure)

Abstract

In this thesis we explore how multiple images from a sequence which individually are considered not usable for a forensic procedure, can be combined to reconstruct a face model that is usable in a forensic face comparison procedure. The best way to improve the current forensic face comparison procedure is to incorporate multiple images in the reconstruction process to reconstruct a high quality frontal view of the face. Based on the available literature and the forensic setting, we chose a structure from motion approach based on landmarks, which seemed the most suitable method for forensic face recognition. The structure from motion method uses landmarks in multiple images to reconstruct the shape of the face and estimates the rotation and translation of the face simultaneously. In the proposed shape reconstruction algorithm, the initial reconstruction is based on a pair of frames with a suitable 2D reprojection error. In an iterative procedure multiple frames were added to improve the 3D reconstruction and the estimates of the rotation and translation of the face in each frame. After the reconstruction of the shape, we triangulated the landmark model to obtain a surface for the texture of the face. Based on the Lambertian illumination model, we corrected and combined the texture from multiple views. The obtained 3D reconstruction is coarse. In the final proposed method we revised the reconstruction of the texture and incorporated a dense shape reconstruction method into the proposed reconstruction method. The dense reconstruction method used the coarse shape reconstruction method as initialization, and creates a dense 3D face reconstruction. The dense reconstruction method is based on an iterative procedure where the normals of the reconstructed face are used to optimize both the shape and the texture of the face. The reconstructed dense 3D models can be used to render frontal faces or faces under small pose. Additional recognition experiments showed that the reconstructed frontal faces outperformed the original non-frontal images in most of the cases. The proposed reconstruction method is unbiased by design and is therefore suitable in a forensic face comparison process.
Original languageEnglish
Awarding Institution
  • University of Twente
Supervisors/Advisors
  • Veldhuis, Raymond N.J., Supervisor
  • Spreeuwers, Lieuwe Jan, Advisor
Award date30 Mar 2017
Place of PublicationEnschede
Publisher
Print ISBNs978-90-365-4324-8
DOIs
Publication statusPublished - 30 Mar 2017

Fingerprint

Image reconstruction
Textures
Face recognition
Lighting
Experiments

Cite this

van Dam, Christiaan. / From image sequence to frontal image : reconstruction of the unknown face: a forensic case. Enschede : University of Twente, 2017. 109 p.
@phdthesis{fff90178000b4d73a7b50e23b3e27d7d,
title = "From image sequence to frontal image: reconstruction of the unknown face: a forensic case",
abstract = "In this thesis we explore how multiple images from a sequence which individually are considered not usable for a forensic procedure, can be combined to reconstruct a face model that is usable in a forensic face comparison procedure. The best way to improve the current forensic face comparison procedure is to incorporate multiple images in the reconstruction process to reconstruct a high quality frontal view of the face. Based on the available literature and the forensic setting, we chose a structure from motion approach based on landmarks, which seemed the most suitable method for forensic face recognition. The structure from motion method uses landmarks in multiple images to reconstruct the shape of the face and estimates the rotation and translation of the face simultaneously. In the proposed shape reconstruction algorithm, the initial reconstruction is based on a pair of frames with a suitable 2D reprojection error. In an iterative procedure multiple frames were added to improve the 3D reconstruction and the estimates of the rotation and translation of the face in each frame. After the reconstruction of the shape, we triangulated the landmark model to obtain a surface for the texture of the face. Based on the Lambertian illumination model, we corrected and combined the texture from multiple views. The obtained 3D reconstruction is coarse. In the final proposed method we revised the reconstruction of the texture and incorporated a dense shape reconstruction method into the proposed reconstruction method. The dense reconstruction method used the coarse shape reconstruction method as initialization, and creates a dense 3D face reconstruction. The dense reconstruction method is based on an iterative procedure where the normals of the reconstructed face are used to optimize both the shape and the texture of the face. The reconstructed dense 3D models can be used to render frontal faces or faces under small pose. Additional recognition experiments showed that the reconstructed frontal faces outperformed the original non-frontal images in most of the cases. The proposed reconstruction method is unbiased by design and is therefore suitable in a forensic face comparison process.",
author = "{van Dam}, Christiaan",
year = "2017",
month = "3",
day = "30",
doi = "10.3990/1.9789036543248",
language = "English",
isbn = "978-90-365-4324-8",
series = "CTIT Ph.D. thesis series",
publisher = "University of Twente",
address = "Netherlands",
school = "University of Twente",

}

From image sequence to frontal image : reconstruction of the unknown face: a forensic case. / van Dam, Christiaan.

Enschede : University of Twente, 2017. 109 p.

Research output: ThesisPhD Thesis - Research UT, graduation UTAcademic

TY - THES

T1 - From image sequence to frontal image

T2 - reconstruction of the unknown face: a forensic case

AU - van Dam, Christiaan

PY - 2017/3/30

Y1 - 2017/3/30

N2 - In this thesis we explore how multiple images from a sequence which individually are considered not usable for a forensic procedure, can be combined to reconstruct a face model that is usable in a forensic face comparison procedure. The best way to improve the current forensic face comparison procedure is to incorporate multiple images in the reconstruction process to reconstruct a high quality frontal view of the face. Based on the available literature and the forensic setting, we chose a structure from motion approach based on landmarks, which seemed the most suitable method for forensic face recognition. The structure from motion method uses landmarks in multiple images to reconstruct the shape of the face and estimates the rotation and translation of the face simultaneously. In the proposed shape reconstruction algorithm, the initial reconstruction is based on a pair of frames with a suitable 2D reprojection error. In an iterative procedure multiple frames were added to improve the 3D reconstruction and the estimates of the rotation and translation of the face in each frame. After the reconstruction of the shape, we triangulated the landmark model to obtain a surface for the texture of the face. Based on the Lambertian illumination model, we corrected and combined the texture from multiple views. The obtained 3D reconstruction is coarse. In the final proposed method we revised the reconstruction of the texture and incorporated a dense shape reconstruction method into the proposed reconstruction method. The dense reconstruction method used the coarse shape reconstruction method as initialization, and creates a dense 3D face reconstruction. The dense reconstruction method is based on an iterative procedure where the normals of the reconstructed face are used to optimize both the shape and the texture of the face. The reconstructed dense 3D models can be used to render frontal faces or faces under small pose. Additional recognition experiments showed that the reconstructed frontal faces outperformed the original non-frontal images in most of the cases. The proposed reconstruction method is unbiased by design and is therefore suitable in a forensic face comparison process.

AB - In this thesis we explore how multiple images from a sequence which individually are considered not usable for a forensic procedure, can be combined to reconstruct a face model that is usable in a forensic face comparison procedure. The best way to improve the current forensic face comparison procedure is to incorporate multiple images in the reconstruction process to reconstruct a high quality frontal view of the face. Based on the available literature and the forensic setting, we chose a structure from motion approach based on landmarks, which seemed the most suitable method for forensic face recognition. The structure from motion method uses landmarks in multiple images to reconstruct the shape of the face and estimates the rotation and translation of the face simultaneously. In the proposed shape reconstruction algorithm, the initial reconstruction is based on a pair of frames with a suitable 2D reprojection error. In an iterative procedure multiple frames were added to improve the 3D reconstruction and the estimates of the rotation and translation of the face in each frame. After the reconstruction of the shape, we triangulated the landmark model to obtain a surface for the texture of the face. Based on the Lambertian illumination model, we corrected and combined the texture from multiple views. The obtained 3D reconstruction is coarse. In the final proposed method we revised the reconstruction of the texture and incorporated a dense shape reconstruction method into the proposed reconstruction method. The dense reconstruction method used the coarse shape reconstruction method as initialization, and creates a dense 3D face reconstruction. The dense reconstruction method is based on an iterative procedure where the normals of the reconstructed face are used to optimize both the shape and the texture of the face. The reconstructed dense 3D models can be used to render frontal faces or faces under small pose. Additional recognition experiments showed that the reconstructed frontal faces outperformed the original non-frontal images in most of the cases. The proposed reconstruction method is unbiased by design and is therefore suitable in a forensic face comparison process.

U2 - 10.3990/1.9789036543248

DO - 10.3990/1.9789036543248

M3 - PhD Thesis - Research UT, graduation UT

SN - 978-90-365-4324-8

T3 - CTIT Ph.D. thesis series

PB - University of Twente

CY - Enschede

ER -

van Dam C. From image sequence to frontal image: reconstruction of the unknown face: a forensic case. Enschede: University of Twente, 2017. 109 p. (CTIT Ph.D. thesis series). https://doi.org/10.3990/1.9789036543248