The Combination and Evaluation of Query Performance Prediction Methods

Claudia Hauff, Leif Azzopardi, Djoerd Hiemstra

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

3 Citations (Scopus)
15 Downloads (Pure)


In this paper, we examine a number of newly applied methods for combining pre-retrieval query performance predictors in order to obtain a better prediction of the query's performance. However, in order to adequately and appropriately compare such techniques, we critically examine the current evaluation methodology and show how using linear correlation coefficients (i) do not provide an intuitive measure indicative of a method's quality, (ii) can provide a misleading indication of performance, and (iii) overstate the performance of combined methods. To address this, we extend the current evaluation methodology to include cross validation, report a more intuitive and descriptive statistic, and apply statistical testing to determine significant differences. During the course of a comprehensive empirical study over several TREC collections, we evaluate nineteen pre-retrieval predictors and three combination methods.
Original languageEnglish
Title of host publicationAdvances in Information Retrieval
Subtitle of host publication31th European Conference on IR Research, ECIR 2009, Toulouse, France, April 6-9, 2009. Proceedings
EditorsMohand Boughanem, Catherine Berrut, Josiane Mothe, Chantal Soule-Dupuy
Place of PublicationBerlin, Heidelberg
Number of pages12
ISBN (Electronic)978-3-642-00958-7
ISBN (Print)978-3-642-00957-0
Publication statusPublished - 2009
Event31th European Conference on Information Retrieval, ECIR 2009: (IR Research) - Toulouse, France
Duration: 6 Apr 20099 Apr 2009
Conference number: 31

Publication series

NameLecture Notes In Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference31th European Conference on Information Retrieval, ECIR 2009
Abbreviated titleECIR


  • METIS-263977
  • CR-H.3.3
  • EWI-15904
  • IR-67852
  • Root Mean Square Error
  • Average Precision
  • Retrieval Performance
  • Query Term
  • Query Performance


Dive into the research topics of 'The Combination and Evaluation of Query Performance Prediction Methods'. Together they form a unique fingerprint.

Cite this