Beyond metadata: searching your archive based on its audio-visual content

T. Tommasi, Robin Aly, K. McGuinness, K. Chatfield, R. Arandjelovic, O. Parkhi, Roeland J.F. Ordelman, A. Zisserman, T. Tuytelaars

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

38 Downloads (Pure)


The EU FP7 project AXES aims at better understanding the needs of archive users and supporting them with systems that reach beyond the state-of-the-art. Our system allows users to instantaneously retrieve content using metadata, spoken words, or a vocabulary of reliably detected visual concepts comprising places, objects and events. Additionally, users can query for new concepts, for which models are learned on-the-fly, using training images obtained from an internet search engine. Thanks to advanced analysis and indexation methods, relevant material can be retrieved within seconds. Our system supports different types of models for object categories (e.g. “bus‿ or “house‿), specific objects (landmarks or logos), person categories (e.g. “people with moustaches‿), or specific persons (e.g. “President Obama‿). Next to text queries, we support query-by-example, which retrieves content containing the same location, objects, or faces shown in provided images. Finally, our system provides alternatives to query-based retrieval by allowing users to browse archives using generated links. Here we evaluate the precision of the retrieved results based on textual queries describing visual content, with the queries extracted from user testing query logs.
Original languageUndefined
Title of host publicationProceedings of the 2014 International Broadcasting Convention, IBC 2014
Place of PublicationStevenage, Herts, UK
PublisherInstitution of Engineering and Technology (IET)
Number of pages3
ISBN (Print)978-1-84919-927-8
Publication statusPublished - Sep 2014

Publication series

PublisherInstitution of Engineering and Technology (IET)


  • audio-visual systems
  • information retrieval systems
  • records management
  • meta data
  • IR-95305
  • METIS-312535
  • EWI-25889

Cite this