Predicting Performance of a Face Recognition System Based on Image Quality

In this dissertation, we focus on several aspects of models that aim to predict performance of a face recognition system. Performance prediction models are commonly based on the following two types of performance predictor features: a) image quality features; and b) features derived solely from similarity scores. We first investigate the merit of these two types of performance predictor features. The evidence from our experiments suggests that the features derived solely from similarity scores are unstable under image quality variations. On the other hand, image quality features have a proven record of being a reliable predictor of face recognition performance. Therefore, the performance prediction model proposed in this dissertation is based only on image quality features. We present a generative model to capture the relation between image quality features q (e. g. pose, illumination, etc) and face recognition performance r (e. g. FMR and FNMR at operating point). Since the model is based only on image quality features, the face recognition performance can be predicted even before the actual recognition has taken place thereby facilitating many preemptive action. A practical limitation of such a data driven generative model is the limited nature of training data set. To address this issue, we have developed a Bayesian ap- proach to model the nature of FNMR and FMR distribution based on the number of match and non-match scores in small regions of the quality space. Random samples drawn from the models provide the initial data essential for training the generative model P (q, r). Experiment results based on six face recognition systems operating on three independent data sets show that the proposed performance prediction model can accurately predict face recognition performance using an accurate and unbiased Image Quality Assessor (IQA). Furthermore, variability in the unaccounted quality space – the image quality features not considered by the IQA – is the major factor causing inaccuracies in predicted performance. Many automatic face recognition systems use automatically detected eye coordinates for facial image registration. We investigate the influence of automatic eye detection error on the performance of face recognition systems. We simulate the error in automatic eye detection by performing facial image registration based on perturbed manually annotated eye coordinates. Since the image quality of probe images are fixed to frontal pose and ambient illumination, the performance variations are solely due to the impact of facial image registration error on face recognition performance. This study helps us understand how image quality variations can amplify its influence on recognition performance by having dual impact on both facial image registration and facial feature extraction/comparison stages of a face recognition system. Our study has shown that, for a face recognition system sensitive to errors in facial image regis- tration, the performance predictor feature set should include some features that can predict the accuracy of automatic eye detector used in the face recognition system. This is essential to accurately model and predict the performance variations in a practical face recognition system. So far, existing work has only focused on using features that predict the performance of face recognition algorithms. Our work has laid the foundation for future work in this direction. A forensic case involving face recognition commonly contains a surveillance view trace (usually a frame from CCTV footage) and a frontal suspect reference set containing facial images of suspects narrowed down by police and forensic investigation. If the forensic investigator chooses to use an automatic face recognition system for this task, there are two choices available: a model based approach or a view based approach. In a model based approach, a frontal view probe image is synthesized based on a 3D model reconstructed from the surveillance view trace. Most face recognition systems are fine tuned for optimal recognition performance for comparing frontal view images and therefore the model based approach, with synthesized frontal probe and frontal suspect reference images, ensures high recognition performance. In a view based approach, the reference set is adapted such that it matches the pose of the surveillance view trace. This approach ensures that a face recognition system always gets to compare facial images under similar pose – not necessarily the frontal view. We investigate if it is potentially more useful to apply a view based approach in foren- sic cases. The evidence from our experiments suggests that the view based approach should be used if: a) it is possible to exactly match the pose, illumination condition and camera of the suspect reference set to that of the probe image (or, forensic trace acquired from CCTV footage); and b) one uses a face recognition system that is capable of comparing non-frontal view facial images with high accuracy. A view based approach may not always be practical because matching pose and camera requires cooperative suspects and access to the same camera that captured the trace image.