From image sequence to frontal image: reconstruction of the unknown face: a forensic case

In this thesis we explore how multiple images from a sequence which individually are considered not usable for a forensic procedure, can be combined to reconstruct a face model that is usable in a forensic face comparison procedure. The best way to improve the current forensic face comparison procedure is to incorporate multiple images in the reconstruction process to reconstruct a high quality frontal view of the face. Based on the available literature and the forensic setting, we chose a structure from motion approach based on landmarks, which seemed the most suitable method for forensic face recognition. The structure from motion method uses landmarks in multiple images to reconstruct the shape of the face and estimates the rotation and translation of the face simultaneously. In the proposed shape reconstruction algorithm, the initial reconstruction is based on a pair of frames with a suitable 2D reprojection error. In an iterative procedure multiple frames were added to improve the 3D reconstruction and the estimates of the rotation and translation of the face in each frame. After the reconstruction of the shape, we triangulated the landmark model to obtain a surface for the texture of the face. Based on the Lambertian illumination model, we corrected and combined the texture from multiple views. The obtained 3D reconstruction is coarse. In the final proposed method we revised the reconstruction of the texture and incorporated a dense shape reconstruction method into the proposed reconstruction method. The dense reconstruction method used the coarse shape reconstruction method as initialization, and creates a dense 3D face reconstruction. The dense reconstruction method is based on an iterative procedure where the normals of the reconstructed face are used to optimize both the shape and the texture of the face. The reconstructed dense 3D models can be used to render frontal faces or faces under small pose. Additional recognition experiments showed that the reconstructed frontal faces outperformed the original non-frontal images in most of the cases. The proposed reconstruction method is unbiased by design and is therefore suitable in a forensic face comparison process.