Merging SVMs with Linear Discriminant Analysis: A Combined Model

S. Nikitidis, S. Zafeiriou, Maja Pantic

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

4 Citations (Scopus)
7 Downloads (Pure)

Abstract

A key problem often encountered by many learning algorithms in computer vision dealing with high dimensional data is the so called "curse of dimensionality" which arises when the available training samples are less than the input feature space dimensionality. To remedy this problem, we propose a joint dimensionality reduction and classification framework by formulating an optimization problem within the maximum margin class separation task. The proposed optimization problem is solved using alternative optimization where we jointly compute the low dimensional maximum margin projections and the separating hyperplanes in the projection subspace. Moreover, in order to reduce the computational cost of the developed optimization algorithm we incorporate orthogonality constraints on the derived projection bases and show that the resulting combined model is an alternation between identifying the optimal separating hyperplanes and performing a linear discriminant analysis on the support vectors. Experiments on face, facial expression and object recognition validate the effectiveness of the proposed method against state-of-the-art dimensionality reduction algorithms.
Original languageUndefined
Title of host publicationProceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014
Place of PublicationUSA
PublisherIEEE Computer Society
Pages1067-1074
Number of pages8
ISBN (Print)978-1-4799-5117-8
DOIs
Publication statusPublished - Jun 2014

Publication series

Name
PublisherIEEE Computer Society
ISSN (Print)1063-6919

Keywords

  • HMI-HF: Human Factors
  • IR-95224
  • METIS-309943
  • EWI-25817

Cite this

Nikitidis, S., Zafeiriou, S., & Pantic, M. (2014). Merging SVMs with Linear Discriminant Analysis: A Combined Model. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014 (pp. 1067-1074). USA: IEEE Computer Society. https://doi.org/10.1109/CVPR.2014.140
Nikitidis, S. ; Zafeiriou, S. ; Pantic, Maja. / Merging SVMs with Linear Discriminant Analysis: A Combined Model. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014. USA : IEEE Computer Society, 2014. pp. 1067-1074
@inproceedings{b5dc78cecbae40fbba8f480d6fca4081,
title = "Merging SVMs with Linear Discriminant Analysis: A Combined Model",
abstract = "A key problem often encountered by many learning algorithms in computer vision dealing with high dimensional data is the so called {"}curse of dimensionality{"} which arises when the available training samples are less than the input feature space dimensionality. To remedy this problem, we propose a joint dimensionality reduction and classification framework by formulating an optimization problem within the maximum margin class separation task. The proposed optimization problem is solved using alternative optimization where we jointly compute the low dimensional maximum margin projections and the separating hyperplanes in the projection subspace. Moreover, in order to reduce the computational cost of the developed optimization algorithm we incorporate orthogonality constraints on the derived projection bases and show that the resulting combined model is an alternation between identifying the optimal separating hyperplanes and performing a linear discriminant analysis on the support vectors. Experiments on face, facial expression and object recognition validate the effectiveness of the proposed method against state-of-the-art dimensionality reduction algorithms.",
keywords = "HMI-HF: Human Factors, IR-95224, METIS-309943, EWI-25817",
author = "S. Nikitidis and S. Zafeiriou and Maja Pantic",
note = "10.1109/CVPR.2014.140",
year = "2014",
month = "6",
doi = "10.1109/CVPR.2014.140",
language = "Undefined",
isbn = "978-1-4799-5117-8",
publisher = "IEEE Computer Society",
pages = "1067--1074",
booktitle = "Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014",
address = "United States",

}

Nikitidis, S, Zafeiriou, S & Pantic, M 2014, Merging SVMs with Linear Discriminant Analysis: A Combined Model. in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014. IEEE Computer Society, USA, pp. 1067-1074. https://doi.org/10.1109/CVPR.2014.140

Merging SVMs with Linear Discriminant Analysis: A Combined Model. / Nikitidis, S.; Zafeiriou, S.; Pantic, Maja.

Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014. USA : IEEE Computer Society, 2014. p. 1067-1074.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Merging SVMs with Linear Discriminant Analysis: A Combined Model

AU - Nikitidis, S.

AU - Zafeiriou, S.

AU - Pantic, Maja

N1 - 10.1109/CVPR.2014.140

PY - 2014/6

Y1 - 2014/6

N2 - A key problem often encountered by many learning algorithms in computer vision dealing with high dimensional data is the so called "curse of dimensionality" which arises when the available training samples are less than the input feature space dimensionality. To remedy this problem, we propose a joint dimensionality reduction and classification framework by formulating an optimization problem within the maximum margin class separation task. The proposed optimization problem is solved using alternative optimization where we jointly compute the low dimensional maximum margin projections and the separating hyperplanes in the projection subspace. Moreover, in order to reduce the computational cost of the developed optimization algorithm we incorporate orthogonality constraints on the derived projection bases and show that the resulting combined model is an alternation between identifying the optimal separating hyperplanes and performing a linear discriminant analysis on the support vectors. Experiments on face, facial expression and object recognition validate the effectiveness of the proposed method against state-of-the-art dimensionality reduction algorithms.

AB - A key problem often encountered by many learning algorithms in computer vision dealing with high dimensional data is the so called "curse of dimensionality" which arises when the available training samples are less than the input feature space dimensionality. To remedy this problem, we propose a joint dimensionality reduction and classification framework by formulating an optimization problem within the maximum margin class separation task. The proposed optimization problem is solved using alternative optimization where we jointly compute the low dimensional maximum margin projections and the separating hyperplanes in the projection subspace. Moreover, in order to reduce the computational cost of the developed optimization algorithm we incorporate orthogonality constraints on the derived projection bases and show that the resulting combined model is an alternation between identifying the optimal separating hyperplanes and performing a linear discriminant analysis on the support vectors. Experiments on face, facial expression and object recognition validate the effectiveness of the proposed method against state-of-the-art dimensionality reduction algorithms.

KW - HMI-HF: Human Factors

KW - IR-95224

KW - METIS-309943

KW - EWI-25817

U2 - 10.1109/CVPR.2014.140

DO - 10.1109/CVPR.2014.140

M3 - Conference contribution

SN - 978-1-4799-5117-8

SP - 1067

EP - 1074

BT - Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014

PB - IEEE Computer Society

CY - USA

ER -

Nikitidis S, Zafeiriou S, Pantic M. Merging SVMs with Linear Discriminant Analysis: A Combined Model. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014. USA: IEEE Computer Society. 2014. p. 1067-1074 https://doi.org/10.1109/CVPR.2014.140