Using 3D Morphable Models for face recognition in video

R.T.A. van Rootseler, L.J. Spreeuwers, R.N.J. Veldhuis

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

    56 Downloads (Pure)

    Abstract

    The 3D Morphable Face Model (3DMM) has been used for over a decade for creating 3D models from single images of faces. This model is based on a PCA model of the 3D shape and texture generated from a limited number of 3D scans. The goal of fitting a 3DMM to an image is to find the model coefficients, the lighting and other imaging variables from which we can remodel that image as accurately as possible. The model coefficients consist of texture and of shape descriptors, and can without further processing be used in verification and recognition experiments. Until now little research has been performed into the influence of the diverse parameters of the 3DMM on the recognition performance. In this paper we will introduce a Bayesian-based method for texture backmapping from multiple images. Using the information from multiple (non-frontal) views we construct a frontal view which can be used as input to 2D face recognition software. We also show how the number of triangles used in the fitting proces influences the recognition performance using the shape descriptors. The verification results of the 3DMM are compared to state-of-the-art 2D face recognition software on the MultiPIE dataset. The 2D FR software outperforms the Morphable Model, but the Morphable Model can be useful as a preprocesser to synthesize a frontal view from a non-frontal view and also combine images with multiple views to a single frontal view. We show results for this preprocessing technique by using an average face shape, a fitted face shape, with a MM texture, with the original texture and with a hybrid texture. The preprocessor has improved the verification results significantly on the dataset.
    Original languageEnglish
    Title of host publicationProceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux
    Place of PublicationEnschede
    PublisherWerkgemeenschap voor Informatie- en Communicatietheorie (WIC)
    Pages235-242
    Number of pages8
    ISBN (Print)978-90-365-3383-6
    Publication statusPublished - May 2012
    Event33rd WIC Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux 2012 - Boekelo, Netherlands
    Duration: 24 May 201225 May 2012
    Conference number: 33

    Conference

    Conference33rd WIC Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux 2012
    CountryNetherlands
    CityBoekelo
    Period24/05/1225/05/12

    Fingerprint

    Face recognition
    Textures
    Lighting
    Imaging techniques

    Keywords

    • METIS-286370
    • IR-80462
    • Video processing
    • SCS-Safety
    • Image Processing
    • Biometrics
    • EWI-21886
    • Morphable Models

    Cite this

    van Rootseler, R. T. A., Spreeuwers, L. J., & Veldhuis, R. N. J. (2012). Using 3D Morphable Models for face recognition in video. In Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux (pp. 235-242). Enschede: Werkgemeenschap voor Informatie- en Communicatietheorie (WIC).
    van Rootseler, R.T.A. ; Spreeuwers, L.J. ; Veldhuis, R.N.J. / Using 3D Morphable Models for face recognition in video. Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Enschede : Werkgemeenschap voor Informatie- en Communicatietheorie (WIC), 2012. pp. 235-242
    @inproceedings{9c579a70def24aeb98696a38858a1333,
    title = "Using 3D Morphable Models for face recognition in video",
    abstract = "The 3D Morphable Face Model (3DMM) has been used for over a decade for creating 3D models from single images of faces. This model is based on a PCA model of the 3D shape and texture generated from a limited number of 3D scans. The goal of fitting a 3DMM to an image is to find the model coefficients, the lighting and other imaging variables from which we can remodel that image as accurately as possible. The model coefficients consist of texture and of shape descriptors, and can without further processing be used in verification and recognition experiments. Until now little research has been performed into the influence of the diverse parameters of the 3DMM on the recognition performance. In this paper we will introduce a Bayesian-based method for texture backmapping from multiple images. Using the information from multiple (non-frontal) views we construct a frontal view which can be used as input to 2D face recognition software. We also show how the number of triangles used in the fitting proces influences the recognition performance using the shape descriptors. The verification results of the 3DMM are compared to state-of-the-art 2D face recognition software on the MultiPIE dataset. The 2D FR software outperforms the Morphable Model, but the Morphable Model can be useful as a preprocesser to synthesize a frontal view from a non-frontal view and also combine images with multiple views to a single frontal view. We show results for this preprocessing technique by using an average face shape, a fitted face shape, with a MM texture, with the original texture and with a hybrid texture. The preprocessor has improved the verification results significantly on the dataset.",
    keywords = "METIS-286370, IR-80462, Video processing, SCS-Safety, Image Processing, Biometrics, EWI-21886, Morphable Models",
    author = "{van Rootseler}, R.T.A. and L.J. Spreeuwers and R.N.J. Veldhuis",
    year = "2012",
    month = "5",
    language = "English",
    isbn = "978-90-365-3383-6",
    pages = "235--242",
    booktitle = "Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux",
    publisher = "Werkgemeenschap voor Informatie- en Communicatietheorie (WIC)",
    address = "Netherlands",

    }

    van Rootseler, RTA, Spreeuwers, LJ & Veldhuis, RNJ 2012, Using 3D Morphable Models for face recognition in video. in Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Werkgemeenschap voor Informatie- en Communicatietheorie (WIC), Enschede, pp. 235-242, 33rd WIC Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux 2012, Boekelo, Netherlands, 24/05/12.

    Using 3D Morphable Models for face recognition in video. / van Rootseler, R.T.A.; Spreeuwers, L.J.; Veldhuis, R.N.J.

    Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Enschede : Werkgemeenschap voor Informatie- en Communicatietheorie (WIC), 2012. p. 235-242.

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

    TY - GEN

    T1 - Using 3D Morphable Models for face recognition in video

    AU - van Rootseler, R.T.A.

    AU - Spreeuwers, L.J.

    AU - Veldhuis, R.N.J.

    PY - 2012/5

    Y1 - 2012/5

    N2 - The 3D Morphable Face Model (3DMM) has been used for over a decade for creating 3D models from single images of faces. This model is based on a PCA model of the 3D shape and texture generated from a limited number of 3D scans. The goal of fitting a 3DMM to an image is to find the model coefficients, the lighting and other imaging variables from which we can remodel that image as accurately as possible. The model coefficients consist of texture and of shape descriptors, and can without further processing be used in verification and recognition experiments. Until now little research has been performed into the influence of the diverse parameters of the 3DMM on the recognition performance. In this paper we will introduce a Bayesian-based method for texture backmapping from multiple images. Using the information from multiple (non-frontal) views we construct a frontal view which can be used as input to 2D face recognition software. We also show how the number of triangles used in the fitting proces influences the recognition performance using the shape descriptors. The verification results of the 3DMM are compared to state-of-the-art 2D face recognition software on the MultiPIE dataset. The 2D FR software outperforms the Morphable Model, but the Morphable Model can be useful as a preprocesser to synthesize a frontal view from a non-frontal view and also combine images with multiple views to a single frontal view. We show results for this preprocessing technique by using an average face shape, a fitted face shape, with a MM texture, with the original texture and with a hybrid texture. The preprocessor has improved the verification results significantly on the dataset.

    AB - The 3D Morphable Face Model (3DMM) has been used for over a decade for creating 3D models from single images of faces. This model is based on a PCA model of the 3D shape and texture generated from a limited number of 3D scans. The goal of fitting a 3DMM to an image is to find the model coefficients, the lighting and other imaging variables from which we can remodel that image as accurately as possible. The model coefficients consist of texture and of shape descriptors, and can without further processing be used in verification and recognition experiments. Until now little research has been performed into the influence of the diverse parameters of the 3DMM on the recognition performance. In this paper we will introduce a Bayesian-based method for texture backmapping from multiple images. Using the information from multiple (non-frontal) views we construct a frontal view which can be used as input to 2D face recognition software. We also show how the number of triangles used in the fitting proces influences the recognition performance using the shape descriptors. The verification results of the 3DMM are compared to state-of-the-art 2D face recognition software on the MultiPIE dataset. The 2D FR software outperforms the Morphable Model, but the Morphable Model can be useful as a preprocesser to synthesize a frontal view from a non-frontal view and also combine images with multiple views to a single frontal view. We show results for this preprocessing technique by using an average face shape, a fitted face shape, with a MM texture, with the original texture and with a hybrid texture. The preprocessor has improved the verification results significantly on the dataset.

    KW - METIS-286370

    KW - IR-80462

    KW - Video processing

    KW - SCS-Safety

    KW - Image Processing

    KW - Biometrics

    KW - EWI-21886

    KW - Morphable Models

    M3 - Conference contribution

    SN - 978-90-365-3383-6

    SP - 235

    EP - 242

    BT - Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux

    PB - Werkgemeenschap voor Informatie- en Communicatietheorie (WIC)

    CY - Enschede

    ER -

    van Rootseler RTA, Spreeuwers LJ, Veldhuis RNJ. Using 3D Morphable Models for face recognition in video. In Proceedings of the 33rd Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux. Enschede: Werkgemeenschap voor Informatie- en Communicatietheorie (WIC). 2012. p. 235-242