Hierarchical online appearance-based tracking for 3D head pose, eyebrows, lips, eyelids, and irises

Javier Orozco, Ognjen Rudovic, Jordi Gonzalez Garcia, Maja Pantic

    Research output: Contribution to journalArticleAcademicpeer-review

    35 Citations (Scopus)
    140 Downloads (Pure)


    In this paper, we propose an On-line Appearance-Based Tracker (OABT) for simultaneous tracking of 3D head pose, lips, eyebrows, eyelids and irises in monocular video sequences. In contrast to previously proposed tracking approaches, which deal with face and gaze tracking separately, our OABT can also be used for eyelid and iris tracking, as well as 3D head pose, lips and eyebrows facial actions tracking. Furthermore, our approach applies an on-line learning of changes in the appearance of the tracked target. Hence, the prior training of appearance models, which usually requires a large amount of labeled facial images, is avoided. Moreover, the proposed method is built upon a hierarchical combination of three OABTs, which are optimized using a Levenberg–Marquardt Algorithm (LMA) enhanced with line-search procedures. This, in turn, makes the proposed method robust to changes in lighting conditions, occlusions and translucent textures, as evidenced by our experiments. Finally, the proposed method achieves head and facial actions tracking in real-time.
    Original languageUndefined
    Pages (from-to)322-340
    Number of pages19
    JournalImage and vision computing
    Issue number4
    Publication statusPublished - Apr 2013


    • IR-89237
    • Levenberg–Marquardt algorithm
    • On-line appearance models
    • Eyelid tracking
    • 3D face tracking
    • Facial action tracking
    • Line-search optimization
    • METIS-302612
    • Iris tracking
    • HMI-HF: Human Factors
    • EWI-24249

    Cite this