Query Performance Prediction: Evaluation Contrasted with Effectiveness

C. Hauff, Leif Azzopardi, Djoerd Hiemstra, Franciska M.G. de Jong

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

11 Citations (Scopus)

Abstract

Query performance predictors are commonly evaluated by reporting correlation coefficients to denote how well the methods perform at predicting the retrieval performance of a set of queries. Despite the amount of research dedicated to this area, one aspect remains neglected: how strong does the correlation need to be in order to realize an improvement in retrieval effectiveness in an operational setting? We address this issue in the context of two settings: Selective Query Expansion and Meta-Search. In an empirical study, we control the quality of a predictor in order to examine how the strength of the correlation achieved, affects the effectiveness of an adaptive retrieval system. The results of this study show that many existing predictors fail to achieve a correlation strong enough to reliably improve the retrieval effectiveness in the Selective Query Expansion as well as the Meta-Search setting.
Original languageUndefined
Title of host publicationAvances in Information Retrieval: Proceedings of the 32nd European Conference on IR Research
Place of PublicationLondon
PublisherSpringer
Pages204-216
Number of pages13
ISBN (Print)978-3-642-12274-3
DOIs
Publication statusPublished - 3 Apr 2010
Event32nd European Conference on IR Research - Milton Keynes, United Kingdom
Duration: 28 Mar 201031 Mar 2010

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Verlag
Volume5993/2010

Conference

Conference32nd European Conference on IR Research
Period28/03/1031/03/10
Other28-31 March 2010

Keywords

  • IR-70843
  • METIS-270784
  • EWI-17780
  • Information Retrieval
  • Evaluation
  • Query performance prediction

Cite this