Fast and exact Newton and Bidirectional fitting of Active Appearance Models

Jean Kossaifi, Georgios Tzimiropoulos, Maja Pantic

  • 1 Citations

Abstract

Active Appearance Models (AAMs) are generative models of shape and appearance that have proven very attractive for their ability to handle wide changes in illumination, pose and occlusion when trained in the wild, while not requiring large training dataset like regression-based or deep learning methods. The problem of fitting an AAM is usually formulated as a non-linear least squares one and the main way of solving it is a standard Gauss-Newton algorithm. In this paper we extend Active Appearance Models in two ways: we first extend the Gauss-Newton framework by formulating a bidirectional fitting method that deforms both the image and the template to fit a new instance. We then formulate a second order method by deriving an efficient Newton method for AAMs fitting. We derive both methods in a unified framework for two types of Active Appearance Models, holistic and part-based, and additionally show how to exploit the structure in the problem to derive fast yet exact solutions. We perform a thorough evaluation of all algorithms on three challenging and recently annotated inthe- wild datasets, and investigate fitting accuracy, convergence properties and the influence of noise in the initialisation. We compare our proposed methods to other algorithms and show that they yield state-of-the-art results, out-performing other methods while having superior convergence properties.
Original languageUndefined
Pages (from-to)1040-1053
Number of pages14
JournalIEEE transactions on image processing
Volume26
Issue number2
DOIs
StatePublished - Feb 2017

Fingerprint

Newton-Raphson method
Lighting

Keywords

  • HMI-HF: Human Factors
  • inverse compositional
  • bidirectional image alignment
  • EC Grant Agreement nr.: FP7/611153
  • IR-104079
  • Active Appearance Models
  • Newton method
  • forward additive
  • EWI-27551

Cite this

Kossaifi, Jean; Tzimiropoulos, Georgios; Pantic, Maja / Fast and exact Newton and Bidirectional fitting of Active Appearance Models.

In: IEEE transactions on image processing, Vol. 26, No. 2, 02.2017, p. 1040-1053.

Research output: Scientific - peer-reviewArticle

@article{fcd16eae87ce4b0b8f5046693de49907,
title = "Fast and exact Newton and Bidirectional fitting of Active Appearance Models",
abstract = "Active Appearance Models (AAMs) are generative models of shape and appearance that have proven very attractive for their ability to handle wide changes in illumination, pose and occlusion when trained in the wild, while not requiring large training dataset like regression-based or deep learning methods. The problem of fitting an AAM is usually formulated as a non-linear least squares one and the main way of solving it is a standard Gauss-Newton algorithm. In this paper we extend Active Appearance Models in two ways: we first extend the Gauss-Newton framework by formulating a bidirectional fitting method that deforms both the image and the template to fit a new instance. We then formulate a second order method by deriving an efficient Newton method for AAMs fitting. We derive both methods in a unified framework for two types of Active Appearance Models, holistic and part-based, and additionally show how to exploit the structure in the problem to derive fast yet exact solutions. We perform a thorough evaluation of all algorithms on three challenging and recently annotated inthe- wild datasets, and investigate fitting accuracy, convergence properties and the influence of noise in the initialisation. We compare our proposed methods to other algorithms and show that they yield state-of-the-art results, out-performing other methods while having superior convergence properties.",
keywords = "HMI-HF: Human Factors, inverse compositional, bidirectional image alignment, EC Grant Agreement nr.: FP7/611153, IR-104079, Active Appearance Models, Newton method, forward additive, EWI-27551",
author = "Jean Kossaifi and Georgios Tzimiropoulos and Maja Pantic",
year = "2017",
month = "2",
doi = "10.1109/TIP.2016.2642828",
volume = "26",
pages = "1040--1053",
journal = "IEEE transactions on image processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

Fast and exact Newton and Bidirectional fitting of Active Appearance Models. / Kossaifi, Jean; Tzimiropoulos, Georgios; Pantic, Maja.

In: IEEE transactions on image processing, Vol. 26, No. 2, 02.2017, p. 1040-1053.

Research output: Scientific - peer-reviewArticle

TY - JOUR

T1 - Fast and exact Newton and Bidirectional fitting of Active Appearance Models

AU - Kossaifi,Jean

AU - Tzimiropoulos,Georgios

AU - Pantic,Maja

PY - 2017/2

Y1 - 2017/2

N2 - Active Appearance Models (AAMs) are generative models of shape and appearance that have proven very attractive for their ability to handle wide changes in illumination, pose and occlusion when trained in the wild, while not requiring large training dataset like regression-based or deep learning methods. The problem of fitting an AAM is usually formulated as a non-linear least squares one and the main way of solving it is a standard Gauss-Newton algorithm. In this paper we extend Active Appearance Models in two ways: we first extend the Gauss-Newton framework by formulating a bidirectional fitting method that deforms both the image and the template to fit a new instance. We then formulate a second order method by deriving an efficient Newton method for AAMs fitting. We derive both methods in a unified framework for two types of Active Appearance Models, holistic and part-based, and additionally show how to exploit the structure in the problem to derive fast yet exact solutions. We perform a thorough evaluation of all algorithms on three challenging and recently annotated inthe- wild datasets, and investigate fitting accuracy, convergence properties and the influence of noise in the initialisation. We compare our proposed methods to other algorithms and show that they yield state-of-the-art results, out-performing other methods while having superior convergence properties.

AB - Active Appearance Models (AAMs) are generative models of shape and appearance that have proven very attractive for their ability to handle wide changes in illumination, pose and occlusion when trained in the wild, while not requiring large training dataset like regression-based or deep learning methods. The problem of fitting an AAM is usually formulated as a non-linear least squares one and the main way of solving it is a standard Gauss-Newton algorithm. In this paper we extend Active Appearance Models in two ways: we first extend the Gauss-Newton framework by formulating a bidirectional fitting method that deforms both the image and the template to fit a new instance. We then formulate a second order method by deriving an efficient Newton method for AAMs fitting. We derive both methods in a unified framework for two types of Active Appearance Models, holistic and part-based, and additionally show how to exploit the structure in the problem to derive fast yet exact solutions. We perform a thorough evaluation of all algorithms on three challenging and recently annotated inthe- wild datasets, and investigate fitting accuracy, convergence properties and the influence of noise in the initialisation. We compare our proposed methods to other algorithms and show that they yield state-of-the-art results, out-performing other methods while having superior convergence properties.

KW - HMI-HF: Human Factors

KW - inverse compositional

KW - bidirectional image alignment

KW - EC Grant Agreement nr.: FP7/611153

KW - IR-104079

KW - Active Appearance Models

KW - Newton method

KW - forward additive

KW - EWI-27551

U2 - 10.1109/TIP.2016.2642828

DO - 10.1109/TIP.2016.2642828

M3 - Article

VL - 26

SP - 1040

EP - 1053

JO - IEEE transactions on image processing

T2 - IEEE transactions on image processing

JF - IEEE transactions on image processing

SN - 1057-7149

IS - 2

ER -