EEG analysis for implicit tagging of video data

Sander Koelstra, C. Mühl, Ioannis Patras

  • 24 Citations

Abstract

In this work, we aim to find neuro-physiological indicators to validate tags attached to video content. Subjects are shown a video and a tag and we aim to determine whether the shown tag was congruent with the presented video by detecting the occurrence of an N400 event-related potential. Tag validation could be used in conjunction with a vision-based recognition system as a feedback mechanism to improve the classification accuracy for multimedia indexing and retrieval. An advantage of using the EEG modality for tag validation is that it is a way of performing implicit tagging. This means it can be performed while the user is passively watching the video. Independent Component Analysis and repeated measures ANOVA are used for analysis. Our experimental results show a clear occurrence of the N400 and a significant difference in N400 activation between matching and non-matching tags.
Original languageUndefined
Title of host publicationProceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. ACII 2009. Volume 2
Place of PublicationLos Alamitos
PublisherIEEE Computer Society Press
Pages27-32
Number of pages6
ISBN (Print)978-1-4244-4799-2
DOIs
StatePublished - 2009
Event3rd International Conference on Affective Computing and Intelligent Interaction, ACII 2009 - Amsterdam, Netherlands

Publication series

Name
PublisherIEEE Computer Society Press

Conference

Conference3rd International Conference on Affective Computing and Intelligent Interaction, ACII 2009
Abbreviated titleACII
CountryNetherlands
CityAmsterdam
Period10/09/0912/09/09

Fingerprint

Independent component analysis
Analysis of variance (ANOVA)
Electroencephalography
Chemical activation
Feedback

Keywords

  • METIS-264313
  • IR-69486
  • multimedia tagging
  • EEG
  • HMI-MI: MULTIMODAL INTERACTIONS
  • Multimedia Retrieval
  • EWI-17167
  • category retrieval
  • HMI-MR: MULTIMEDIA RETRIEVAL
  • N400

Cite this

Koelstra, S., Mühl, C., & Patras, I. (2009). EEG analysis for implicit tagging of video data. In Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. ACII 2009. Volume 2 (pp. 27-32). [10.1109/ACII.2009.5349482] Los Alamitos: IEEE Computer Society Press. DOI: 10.1109/ACII.2009.5349482

Koelstra, Sander; Mühl, C.; Patras, Ioannis / EEG analysis for implicit tagging of video data.

Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. ACII 2009. Volume 2. Los Alamitos : IEEE Computer Society Press, 2009. p. 27-32 10.1109/ACII.2009.5349482.

Research output: Scientific - peer-reviewConference contribution

@inbook{5b0fee39472d4607b6a9e188ded10d0c,
title = "EEG analysis for implicit tagging of video data",
abstract = "In this work, we aim to find neuro-physiological indicators to validate tags attached to video content. Subjects are shown a video and a tag and we aim to determine whether the shown tag was congruent with the presented video by detecting the occurrence of an N400 event-related potential. Tag validation could be used in conjunction with a vision-based recognition system as a feedback mechanism to improve the classification accuracy for multimedia indexing and retrieval. An advantage of using the EEG modality for tag validation is that it is a way of performing implicit tagging. This means it can be performed while the user is passively watching the video. Independent Component Analysis and repeated measures ANOVA are used for analysis. Our experimental results show a clear occurrence of the N400 and a significant difference in N400 activation between matching and non-matching tags.",
keywords = "METIS-264313, IR-69486, multimedia tagging, EEG, HMI-MI: MULTIMODAL INTERACTIONS, Multimedia Retrieval, EWI-17167, category retrieval, HMI-MR: MULTIMEDIA RETRIEVAL, N400",
author = "Sander Koelstra and C. Mühl and Ioannis Patras",
note = "10.1109/ACII.2009.5349482",
year = "2009",
doi = "10.1109/ACII.2009.5349482",
isbn = "978-1-4244-4799-2",
publisher = "IEEE Computer Society Press",
pages = "27--32",
booktitle = "Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. ACII 2009. Volume 2",

}

Koelstra, S, Mühl, C & Patras, I 2009, EEG analysis for implicit tagging of video data. in Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. ACII 2009. Volume 2., 10.1109/ACII.2009.5349482, IEEE Computer Society Press, Los Alamitos, pp. 27-32, 3rd International Conference on Affective Computing and Intelligent Interaction, ACII 2009, Amsterdam, Netherlands, 10-12 September. DOI: 10.1109/ACII.2009.5349482

EEG analysis for implicit tagging of video data. / Koelstra, Sander; Mühl, C.; Patras, Ioannis.

Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. ACII 2009. Volume 2. Los Alamitos : IEEE Computer Society Press, 2009. p. 27-32 10.1109/ACII.2009.5349482.

Research output: Scientific - peer-reviewConference contribution

TY - CHAP

T1 - EEG analysis for implicit tagging of video data

AU - Koelstra,Sander

AU - Mühl,C.

AU - Patras,Ioannis

N1 - 10.1109/ACII.2009.5349482

PY - 2009

Y1 - 2009

N2 - In this work, we aim to find neuro-physiological indicators to validate tags attached to video content. Subjects are shown a video and a tag and we aim to determine whether the shown tag was congruent with the presented video by detecting the occurrence of an N400 event-related potential. Tag validation could be used in conjunction with a vision-based recognition system as a feedback mechanism to improve the classification accuracy for multimedia indexing and retrieval. An advantage of using the EEG modality for tag validation is that it is a way of performing implicit tagging. This means it can be performed while the user is passively watching the video. Independent Component Analysis and repeated measures ANOVA are used for analysis. Our experimental results show a clear occurrence of the N400 and a significant difference in N400 activation between matching and non-matching tags.

AB - In this work, we aim to find neuro-physiological indicators to validate tags attached to video content. Subjects are shown a video and a tag and we aim to determine whether the shown tag was congruent with the presented video by detecting the occurrence of an N400 event-related potential. Tag validation could be used in conjunction with a vision-based recognition system as a feedback mechanism to improve the classification accuracy for multimedia indexing and retrieval. An advantage of using the EEG modality for tag validation is that it is a way of performing implicit tagging. This means it can be performed while the user is passively watching the video. Independent Component Analysis and repeated measures ANOVA are used for analysis. Our experimental results show a clear occurrence of the N400 and a significant difference in N400 activation between matching and non-matching tags.

KW - METIS-264313

KW - IR-69486

KW - multimedia tagging

KW - EEG

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - Multimedia Retrieval

KW - EWI-17167

KW - category retrieval

KW - HMI-MR: MULTIMEDIA RETRIEVAL

KW - N400

U2 - 10.1109/ACII.2009.5349482

DO - 10.1109/ACII.2009.5349482

M3 - Conference contribution

SN - 978-1-4244-4799-2

SP - 27

EP - 32

BT - Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. ACII 2009. Volume 2

PB - IEEE Computer Society Press

ER -

Koelstra S, Mühl C, Patras I. EEG analysis for implicit tagging of video data. In Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. ACII 2009. Volume 2. Los Alamitos: IEEE Computer Society Press. 2009. p. 27-32. 10.1109/ACII.2009.5349482. Available from, DOI: 10.1109/ACII.2009.5349482