A unified framework for probabilistic component analysis

Mihalis A. Nicolaou, Stefanos Zafeiriou, Maja Pantic

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    2 Citations (Scopus)
    12 Downloads (Pure)


    We present a unifying framework which reduces the construction of probabilistic component analysis techniques to a mere selection of the latent neighbourhood, thus providing an elegant and principled framework for creating novel component analysis models as well as constructing probabilistic equivalents of deterministic component analysis methods. Under our framework, we unify many very popular and well-studied component analysis algorithms, such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Locality Preserving Projections (LPP) and Slow Feature Analysis (SFA), some of which have no probabilistic equivalents in literature thus far. We firstly define the Markov Random Fields (MRFs) which encapsulate the latent connectivity of the aforementioned component analysis techniques; subsequently, we show that the projection directions produced by all PCA, LDA, LPP and SFA are also produced by the Maximum Likelihood (ML) solution of a single joint probability density function, composed by selecting one of the defined MRF priors while utilising a simple observation model. Furthermore, we propose novel Expectation Maximization (EM) algorithms, exploiting the proposed joint PDF, while we generalize the proposed methodologies to arbitrary connectivities via parametrizable MRF products. Theoretical analysis and experiments on both simulated and real world data show the usefulness of the proposed framework, by deriving methods which well outperform state-of-the-art equivalents.
    Original languageUndefined
    Title of host publicationProceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2014
    EditorsToon Calders, Floriana Esposito, Eyke Hüllermeier, Rosa Meo
    Place of PublicationBerlin
    Number of pages16
    ISBN (Print)978-3-662-44850-2
    Publication statusPublished - Sep 2014
    EventEuropean Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2014 - Nancy, France
    Duration: 15 Sep 201419 Sep 2014

    Publication series

    NameLecture Notes in Computer Science
    PublisherSpringer Verlag
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349


    ConferenceEuropean Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2014
    Other15-19 September 2014


    • HMI-HF: Human Factors
    • EC Grant Agreement nr.: FP7/288235
    • EWI-25812
    • EC Grant Agreement nr.: FP7/2007-2013
    • Dimensionality Reduction
    • METIS-309931
    • Random Fields
    • Probabilistic Methods
    • IR-94679
    • Component Analysis
    • Unifying Framework

    Cite this