Multimedia implicit tagging using EEG signals

M. Soleymani, Maja Pantic

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

9 Citations (Scopus)
52 Downloads (Pure)

Abstract

Electroencephalogram (EEG) signals reflect brain activities associated with emotional and cognitive processes. In this paper, we demonstrate how they can be used to find tags for multimedia content without users' direct input. Alternative methods for multimedia tagging is attracting increasing interest from multimedia community. The new portable EEG helmets are paving the way for employing brain waves in human computer interaction. In this paper, we demonstrate the performance of EEG for tagging purposes using two different scenarios on MAHNOB-HCI database. First, an emotional tagging and classification using a reduced set of electrodes is presented. The emotional responses of 24 participants to short video clips are classified into three classes on arousal and valence. We show how a reduced set of electrodes based on previous studies can preserve and even enhance the emotional classification rate. We then demonstrate the feasibility of using EEG signals for tag relevance tasks. A set of images were shown to participants first, without any tag and then with a relevant or irrelevant tag. The relevance of the tag was assessed based on the EEG responses of the participants in the first second after the tag was depicted. Finally, we demonstrate that by aggregating multiple participants' responses we can significantly improve the tagging accuracy.
Original languageEnglish
Title of host publicationProceedings of the IEEE International Conference on Multimedia and Expo, ICME 2013
Place of PublicationUSA
PublisherIEEE Computer Society
Pages1-6
Number of pages6
ISBN (Print)978-1-4799-0015-2
DOIs
Publication statusPublished - Jul 2013
EventIEEE International Conference on Multimedia and Expo, ICME 2013 - San Jose, United States
Duration: 15 Jul 201319 Jul 2013

Publication series

Name
PublisherIEEE Computer Society
ISSN (Print)1945-7871

Conference

ConferenceIEEE International Conference on Multimedia and Expo, ICME 2013
Abbreviated titleICME
CountryUnited States
CitySan Jose
Period15/07/1319/07/13

Fingerprint

Electroencephalography
Human computer interaction
Brain
Electrodes

Keywords

  • EWI-24326
  • HMI-HF: Human Factors
  • EEG
  • tag relevance
  • IR-89321
  • Implicit Tagging
  • Multi-media
  • METIS-302653
  • Affect

Cite this

Soleymani, M., & Pantic, M. (2013). Multimedia implicit tagging using EEG signals. In Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2013 (pp. 1-6). USA: IEEE Computer Society. https://doi.org/10.1109/ICME.2013.6607623
Soleymani, M. ; Pantic, Maja. / Multimedia implicit tagging using EEG signals. Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2013. USA : IEEE Computer Society, 2013. pp. 1-6
@inproceedings{8975389441484445a1dfc02a1ea1f6ed,
title = "Multimedia implicit tagging using EEG signals",
abstract = "Electroencephalogram (EEG) signals reflect brain activities associated with emotional and cognitive processes. In this paper, we demonstrate how they can be used to find tags for multimedia content without users' direct input. Alternative methods for multimedia tagging is attracting increasing interest from multimedia community. The new portable EEG helmets are paving the way for employing brain waves in human computer interaction. In this paper, we demonstrate the performance of EEG for tagging purposes using two different scenarios on MAHNOB-HCI database. First, an emotional tagging and classification using a reduced set of electrodes is presented. The emotional responses of 24 participants to short video clips are classified into three classes on arousal and valence. We show how a reduced set of electrodes based on previous studies can preserve and even enhance the emotional classification rate. We then demonstrate the feasibility of using EEG signals for tag relevance tasks. A set of images were shown to participants first, without any tag and then with a relevant or irrelevant tag. The relevance of the tag was assessed based on the EEG responses of the participants in the first second after the tag was depicted. Finally, we demonstrate that by aggregating multiple participants' responses we can significantly improve the tagging accuracy.",
keywords = "EWI-24326, HMI-HF: Human Factors, EEG, tag relevance, IR-89321, Implicit Tagging, Multi-media, METIS-302653, Affect",
author = "M. Soleymani and Maja Pantic",
note = "10.1109/ICME.2013.6607623",
year = "2013",
month = "7",
doi = "10.1109/ICME.2013.6607623",
language = "English",
isbn = "978-1-4799-0015-2",
publisher = "IEEE Computer Society",
pages = "1--6",
booktitle = "Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2013",
address = "United States",

}

Soleymani, M & Pantic, M 2013, Multimedia implicit tagging using EEG signals. in Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2013. IEEE Computer Society, USA, pp. 1-6, IEEE International Conference on Multimedia and Expo, ICME 2013, San Jose, United States, 15/07/13. https://doi.org/10.1109/ICME.2013.6607623

Multimedia implicit tagging using EEG signals. / Soleymani, M.; Pantic, Maja.

Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2013. USA : IEEE Computer Society, 2013. p. 1-6.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Multimedia implicit tagging using EEG signals

AU - Soleymani, M.

AU - Pantic, Maja

N1 - 10.1109/ICME.2013.6607623

PY - 2013/7

Y1 - 2013/7

N2 - Electroencephalogram (EEG) signals reflect brain activities associated with emotional and cognitive processes. In this paper, we demonstrate how they can be used to find tags for multimedia content without users' direct input. Alternative methods for multimedia tagging is attracting increasing interest from multimedia community. The new portable EEG helmets are paving the way for employing brain waves in human computer interaction. In this paper, we demonstrate the performance of EEG for tagging purposes using two different scenarios on MAHNOB-HCI database. First, an emotional tagging and classification using a reduced set of electrodes is presented. The emotional responses of 24 participants to short video clips are classified into three classes on arousal and valence. We show how a reduced set of electrodes based on previous studies can preserve and even enhance the emotional classification rate. We then demonstrate the feasibility of using EEG signals for tag relevance tasks. A set of images were shown to participants first, without any tag and then with a relevant or irrelevant tag. The relevance of the tag was assessed based on the EEG responses of the participants in the first second after the tag was depicted. Finally, we demonstrate that by aggregating multiple participants' responses we can significantly improve the tagging accuracy.

AB - Electroencephalogram (EEG) signals reflect brain activities associated with emotional and cognitive processes. In this paper, we demonstrate how they can be used to find tags for multimedia content without users' direct input. Alternative methods for multimedia tagging is attracting increasing interest from multimedia community. The new portable EEG helmets are paving the way for employing brain waves in human computer interaction. In this paper, we demonstrate the performance of EEG for tagging purposes using two different scenarios on MAHNOB-HCI database. First, an emotional tagging and classification using a reduced set of electrodes is presented. The emotional responses of 24 participants to short video clips are classified into three classes on arousal and valence. We show how a reduced set of electrodes based on previous studies can preserve and even enhance the emotional classification rate. We then demonstrate the feasibility of using EEG signals for tag relevance tasks. A set of images were shown to participants first, without any tag and then with a relevant or irrelevant tag. The relevance of the tag was assessed based on the EEG responses of the participants in the first second after the tag was depicted. Finally, we demonstrate that by aggregating multiple participants' responses we can significantly improve the tagging accuracy.

KW - EWI-24326

KW - HMI-HF: Human Factors

KW - EEG

KW - tag relevance

KW - IR-89321

KW - Implicit Tagging

KW - Multi-media

KW - METIS-302653

KW - Affect

U2 - 10.1109/ICME.2013.6607623

DO - 10.1109/ICME.2013.6607623

M3 - Conference contribution

SN - 978-1-4799-0015-2

SP - 1

EP - 6

BT - Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2013

PB - IEEE Computer Society

CY - USA

ER -

Soleymani M, Pantic M. Multimedia implicit tagging using EEG signals. In Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2013. USA: IEEE Computer Society. 2013. p. 1-6 https://doi.org/10.1109/ICME.2013.6607623