Robust correlated and individual component analysis

Yannis Panagakis, Mihalis A. Nicolaou, Stefanos Zafeiriou, Maja Pantic

    Research output: Contribution to journalArticleAcademicpeer-review

    32 Citations (Scopus)


    Recovering correlated and individual components of two, possibly temporally misaligned, sets of data is a fundamental task in disciplines such as image, vision, and behavior computing, with application to problems such as multi-modal fusion (via correlated components), predictive analysis, and clustering (via the individual ones). Here, we study the extraction of correlated and individual components under real-world conditions, namely i) the presence of gross non-Gaussian noise and ii) temporally misaligned data. In this light, we propose a method for the Robust Correlated and Individual Component Analysis (RCICA) of two sets of data in the presence of gross, sparse errors. We furthermore extend RCICA in order to handle temporal incongruities arising in the data. To this end, two suitable optimization problems are solved. The generality of the proposed methods is demonstrated by applying them onto 4 applications, namely i) heterogeneous face recognition, ii) multi-modal feature fusion for human behavior analysis (i.e., audio-visual prediction of interest and conflict), iii) face clustering, and iv) thetemporal alignment of facial expressions. Experimental results on 2 synthetic and 7 real world datasets indicate the robustness and effectiveness of the proposed methodson these application domains, outperforming other state-of-the-art methods in the field. © 1979-2012 IEEE.
    Original languageUndefined
    Pages (from-to)1665-1678
    Number of pages14
    JournalIEEE transactions on pattern analysis and machine intelligence
    Issue number8
    Publication statusPublished - 1 Aug 2016


    • EWI-27126
    • HMI-HF: Human Factors
    • Sparsity
    • canonical correlation analysis
    • Low rank
    • time warping
    • EC Grant Agreement nr.: FP7/611153
    • IR-103791
    • Multi-modal analysis
    • individual components

    Cite this