Average Precision: Good Guide or False Friend to Multimedia Search Effectiveness?

Robin Aly, Rudolf Berend Trieschnigg, Kevin McGuinness, Noel E. O' Connor, Franciska M.G. de Jong

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)
9 Downloads (Pure)

Abstract

Approaches to multimedia search often evolve from existing approaches with strong average precision. However, work on search evaluation shows that average precision does not always capture effectiveness in terms of satisfying user needs because it ignores the diversity of search results. This paper investigates whether search approaches with diverse results have been neglected within the multimedia retrieval research agenda due the fact that they are overshadowed by search approaches with strong average precision. To this end, we compare 361 search approaches applied on the TrecVid benchmarks between 2005 and 2007. We motivate two criteria based on measure correlation and statistical equivalence to estimate whether search approaches with diverse results have been neglected. We show that hypothesized effect indeed occurs in the above examined collections. As a consequence, the research community would benefit from reconsidering existing approaches in the light of diversity.
Original languageUndefined
Title of host publicationProceedings of the 20th Anniversary International Conference on MultiMedia Modelling, MMM 2014, Part II
Place of PublicationLondon
PublisherSpringer
Pages239-250
Number of pages12
ISBN (Print)978-3-319-04116-2
DOIs
Publication statusPublished - Jan 2014

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Verlag
Volume8326
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords

  • EWI-25893
  • METIS-312538
  • IR-95308

Cite this

Aly, R., Trieschnigg, R. B., McGuinness, K., O' Connor, N. E., & de Jong, F. M. G. (2014). Average Precision: Good Guide or False Friend to Multimedia Search Effectiveness? In Proceedings of the 20th Anniversary International Conference on MultiMedia Modelling, MMM 2014, Part II (pp. 239-250). (Lecture Notes in Computer Science; Vol. 8326). London: Springer. https://doi.org/10.1007/978-3-319-04117-9_22