A system for the semantic multimodal analysis of news audio-visual content

Vasileios Mezaris, Spyros Gidaros, Georgios Th. Papadopoulos, Walter Kasper, Roeland J.F. Ordelman, Jörg Steffen, M.A.H. Huijbregts, Franciska M.G. de Jong, Ioannis Kompatsiaris, Michael G. Strintzis

    Research output: Contribution to journalArticleAcademicpeer-review

    8 Citations (Scopus)

    Abstract

    News-related content is nowadays among the most popular types of content for users in everyday applications. Although the generation and distribution of news content has become commonplace, due to the availability of inexpensive media capturing devices and the development of media sharing services targeting both professional and user-generated news content, the automatic analysis and annotation that is required for supporting intelligent search and delivery of this content remains an open issue. In this paper, a complete architecture for knowledge-assisted multimodal analysis of news-related multimedia content is presented, along with its constituent components. The proposed analysis architecture employs state-of-the-art methods for the analysis of each individual modality (visual, audio, text) separately and proposes a novel fusion technique based on the particular characteristics of news-related content for the combination of the individual modality analysis results. Experimental results on news broadcast video illustrate the usefulness of the proposed techniques in the automatic generation of semantic annotations.
    Original languageUndefined
    Pages (from-to)645052
    Number of pages16
    JournalEURASIP journal on advances in signal processing
    Volume2010
    Issue number47
    DOIs
    Publication statusPublished - Feb 2010

    Keywords

    • EWI-19888
    • IR-76505
    • METIS-276406

    Cite this

    Mezaris, V., Gidaros, S., Papadopoulos, G. T., Kasper, W., Ordelman, R. J. F., Steffen, J., ... Strintzis, M. G. (2010). A system for the semantic multimodal analysis of news audio-visual content. EURASIP journal on advances in signal processing, 2010(47), 645052. https://doi.org/10.1155/2010/645052
    Mezaris, Vasileios ; Gidaros, Spyros ; Papadopoulos, Georgios Th. ; Kasper, Walter ; Ordelman, Roeland J.F. ; Steffen, Jörg ; Huijbregts, M.A.H. ; de Jong, Franciska M.G. ; Kompatsiaris, Ioannis ; Strintzis, Michael G. / A system for the semantic multimodal analysis of news audio-visual content. In: EURASIP journal on advances in signal processing. 2010 ; Vol. 2010, No. 47. pp. 645052.
    @article{78f937c30c8d4375b1cb543aa4498633,
    title = "A system for the semantic multimodal analysis of news audio-visual content",
    abstract = "News-related content is nowadays among the most popular types of content for users in everyday applications. Although the generation and distribution of news content has become commonplace, due to the availability of inexpensive media capturing devices and the development of media sharing services targeting both professional and user-generated news content, the automatic analysis and annotation that is required for supporting intelligent search and delivery of this content remains an open issue. In this paper, a complete architecture for knowledge-assisted multimodal analysis of news-related multimedia content is presented, along with its constituent components. The proposed analysis architecture employs state-of-the-art methods for the analysis of each individual modality (visual, audio, text) separately and proposes a novel fusion technique based on the particular characteristics of news-related content for the combination of the individual modality analysis results. Experimental results on news broadcast video illustrate the usefulness of the proposed techniques in the automatic generation of semantic annotations.",
    keywords = "EWI-19888, IR-76505, METIS-276406",
    author = "Vasileios Mezaris and Spyros Gidaros and Papadopoulos, {Georgios Th.} and Walter Kasper and Ordelman, {Roeland J.F.} and J{\"o}rg Steffen and M.A.H. Huijbregts and {de Jong}, {Franciska M.G.} and Ioannis Kompatsiaris and Strintzis, {Michael G.}",
    note = "10.1155/2010/645052",
    year = "2010",
    month = "2",
    doi = "10.1155/2010/645052",
    language = "Undefined",
    volume = "2010",
    pages = "645052",
    journal = "EURASIP journal on advances in signal processing",
    issn = "1687-6172",
    publisher = "Springer",
    number = "47",

    }

    Mezaris, V, Gidaros, S, Papadopoulos, GT, Kasper, W, Ordelman, RJF, Steffen, J, Huijbregts, MAH, de Jong, FMG, Kompatsiaris, I & Strintzis, MG 2010, 'A system for the semantic multimodal analysis of news audio-visual content', EURASIP journal on advances in signal processing, vol. 2010, no. 47, pp. 645052. https://doi.org/10.1155/2010/645052

    A system for the semantic multimodal analysis of news audio-visual content. / Mezaris, Vasileios; Gidaros, Spyros; Papadopoulos, Georgios Th.; Kasper, Walter; Ordelman, Roeland J.F.; Steffen, Jörg; Huijbregts, M.A.H.; de Jong, Franciska M.G.; Kompatsiaris, Ioannis; Strintzis, Michael G.

    In: EURASIP journal on advances in signal processing, Vol. 2010, No. 47, 02.2010, p. 645052.

    Research output: Contribution to journalArticleAcademicpeer-review

    TY - JOUR

    T1 - A system for the semantic multimodal analysis of news audio-visual content

    AU - Mezaris, Vasileios

    AU - Gidaros, Spyros

    AU - Papadopoulos, Georgios Th.

    AU - Kasper, Walter

    AU - Ordelman, Roeland J.F.

    AU - Steffen, Jörg

    AU - Huijbregts, M.A.H.

    AU - de Jong, Franciska M.G.

    AU - Kompatsiaris, Ioannis

    AU - Strintzis, Michael G.

    N1 - 10.1155/2010/645052

    PY - 2010/2

    Y1 - 2010/2

    N2 - News-related content is nowadays among the most popular types of content for users in everyday applications. Although the generation and distribution of news content has become commonplace, due to the availability of inexpensive media capturing devices and the development of media sharing services targeting both professional and user-generated news content, the automatic analysis and annotation that is required for supporting intelligent search and delivery of this content remains an open issue. In this paper, a complete architecture for knowledge-assisted multimodal analysis of news-related multimedia content is presented, along with its constituent components. The proposed analysis architecture employs state-of-the-art methods for the analysis of each individual modality (visual, audio, text) separately and proposes a novel fusion technique based on the particular characteristics of news-related content for the combination of the individual modality analysis results. Experimental results on news broadcast video illustrate the usefulness of the proposed techniques in the automatic generation of semantic annotations.

    AB - News-related content is nowadays among the most popular types of content for users in everyday applications. Although the generation and distribution of news content has become commonplace, due to the availability of inexpensive media capturing devices and the development of media sharing services targeting both professional and user-generated news content, the automatic analysis and annotation that is required for supporting intelligent search and delivery of this content remains an open issue. In this paper, a complete architecture for knowledge-assisted multimodal analysis of news-related multimedia content is presented, along with its constituent components. The proposed analysis architecture employs state-of-the-art methods for the analysis of each individual modality (visual, audio, text) separately and proposes a novel fusion technique based on the particular characteristics of news-related content for the combination of the individual modality analysis results. Experimental results on news broadcast video illustrate the usefulness of the proposed techniques in the automatic generation of semantic annotations.

    KW - EWI-19888

    KW - IR-76505

    KW - METIS-276406

    U2 - 10.1155/2010/645052

    DO - 10.1155/2010/645052

    M3 - Article

    VL - 2010

    SP - 645052

    JO - EURASIP journal on advances in signal processing

    JF - EURASIP journal on advances in signal processing

    SN - 1687-6172

    IS - 47

    ER -