Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment

Victor de Boer; Roeland J.F. Ordelman; Josefien Schuurman

doi:10.1007/978-3-319-24592-8_4

Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment

Victor de Boer, Roeland J.F. Ordelman, Josefien Schuurman

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

22 Downloads (Pure)

Abstract

In this paper we report on an evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion are sufficiently high. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and also assess the effect of the transformation from manual to automatic from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.

Original language	Undefined
Title of host publication	Research and Advanced Technology for Digital Libraries, Proceedings of the 19th International Conference on Theory and Practice of Digital Libraries, TPDL 2015
Place of Publication	Switzerland
Publisher	Springer
Pages	43-55
Number of pages	13
ISBN (Print)	978-3-319-24591-1
DOIs	https://doi.org/10.1007/978-3-319-24592-8_4
Publication status	Published - Sept 2015
Event	19th International Conference on Theory and Practice of Digital Libraries 2015: Connecting Digital Collections - Lecture-Conference Center, Poznan, Poland Duration: 14 Sept 2015 → 18 Sept 2015 Conference number: 19 http://tpdl2015.info/

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer International Publishing
Volume	9316
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	19th International Conference on Theory and Practice of Digital Libraries 2015
Abbreviated title	TPDL 2015
Country/Territory	Poland
City	Poznan
Period	14/09/15 → 18/09/15
Internet address	http://tpdl2015.info/

Keywords

EWI-26373
HMI-MR: MULTIMEDIA RETRIEVAL
Thesaurus
Practice-oriented evaluation
METIS-314983
Audiovisual access
Audiovisual archives
Information Extraction
IR-98283

Access to Document

10.1007/978-3-319-24592-8_4

chp%3A10.1007%2F978-3-319-24592-8_4

Cite this

de Boer, V., Ordelman, R. J. F., & Schuurman, J. (2015). Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment. In Research and Advanced Technology for Digital Libraries, Proceedings of the 19th International Conference on Theory and Practice of Digital Libraries, TPDL 2015 (pp. 43-55). (Lecture Notes in Computer Science; Vol. 9316). Springer. https://doi.org/10.1007/978-3-319-24592-8_4

de Boer, Victor ; Ordelman, Roeland J.F. ; Schuurman, Josefien. / Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment. Research and Advanced Technology for Digital Libraries, Proceedings of the 19th International Conference on Theory and Practice of Digital Libraries, TPDL 2015. Switzerland : Springer, 2015. pp. 43-55 (Lecture Notes in Computer Science).

@inproceedings{4d6f60f91e134146a9250fc06c8f7e68,

title = "Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment",

abstract = "In this paper we report on an evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion are sufficiently high. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and also assess the effect of the transformation from manual to automatic from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.",

keywords = "EWI-26373, HMI-MR: MULTIMEDIA RETRIEVAL, Thesaurus, Practice-oriented evaluation, METIS-314983, Audiovisual access, Audiovisual archives, Information Extraction, IR-98283",

author = "{de Boer}, Victor and Ordelman, {Roeland J.F.} and Josefien Schuurman",

note = "10.1007/978-3-319-24592-8_4 ; 19th International Conference on Theory and Practice of Digital Libraries 2015 ; Conference date: 14-09-2015 Through 18-09-2015",

year = "2015",

month = sep,

doi = "10.1007/978-3-319-24592-8_4",

language = "Undefined",

isbn = "978-3-319-24591-1",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "43--55",

booktitle = "Research and Advanced Technology for Digital Libraries, Proceedings of the 19th International Conference on Theory and Practice of Digital Libraries, TPDL 2015",

address = "Germany",

url = "http://tpdl2015.info/",

}

de Boer, V, Ordelman, RJF & Schuurman, J 2015, Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment. in Research and Advanced Technology for Digital Libraries, Proceedings of the 19th International Conference on Theory and Practice of Digital Libraries, TPDL 2015. Lecture Notes in Computer Science, vol. 9316, Springer, Switzerland, pp. 43-55, 19th International Conference on Theory and Practice of Digital Libraries 2015, Poznan, Poland, 14/09/15. https://doi.org/10.1007/978-3-319-24592-8_4

Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment. / de Boer, Victor; Ordelman, Roeland J.F.; Schuurman, Josefien.
Research and Advanced Technology for Digital Libraries, Proceedings of the 19th International Conference on Theory and Practice of Digital Libraries, TPDL 2015. Switzerland: Springer, 2015. p. 43-55 (Lecture Notes in Computer Science; Vol. 9316).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment

AU - de Boer, Victor

AU - Ordelman, Roeland J.F.

AU - Schuurman, Josefien

N1 - Conference code: 19

PY - 2015/9

Y1 - 2015/9

N2 - In this paper we report on an evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion are sufficiently high. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and also assess the effect of the transformation from manual to automatic from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.

AB - In this paper we report on an evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion are sufficiently high. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and also assess the effect of the transformation from manual to automatic from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.

KW - EWI-26373

KW - HMI-MR: MULTIMEDIA RETRIEVAL

KW - Thesaurus

KW - Practice-oriented evaluation

KW - METIS-314983

KW - Audiovisual access

KW - Audiovisual archives

KW - Information Extraction

KW - IR-98283

U2 - 10.1007/978-3-319-24592-8_4

DO - 10.1007/978-3-319-24592-8_4

M3 - Conference contribution

SN - 978-3-319-24591-1

T3 - Lecture Notes in Computer Science

SP - 43

EP - 55

BT - Research and Advanced Technology for Digital Libraries, Proceedings of the 19th International Conference on Theory and Practice of Digital Libraries, TPDL 2015

PB - Springer

CY - Switzerland

T2 - 19th International Conference on Theory and Practice of Digital Libraries 2015

Y2 - 14 September 2015 through 18 September 2015

ER -

de Boer V, Ordelman RJF, Schuurman J. Practice-Oriented Evaluation of Unsupervised Labeling of Audiovisual Content in an Archive Production Environment. In Research and Advanced Technology for Digital Libraries, Proceedings of the 19th International Conference on Theory and Practice of Digital Libraries, TPDL 2015. Switzerland: Springer. 2015. p. 43-55. (Lecture Notes in Computer Science). doi: 10.1007/978-3-319-24592-8_4