When is query performance prediction effective?

C. Hauff, L. Azzopardi

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    4 Citations (Scopus)

    Abstract

    The utility of Query Performance Prediction (QPP) methods is commonly evaluated by reporting correlation coefficients to denote how well the methods perform at predicting the retrieval performance of a set of queries. However, a quintessential question remains unexplored: how strong does the correlation need to be in order to realize an increase in retrieval performance? In this work, we address this question in the context of Selective Query Expansion (SQE) and perform a large-scale experiment. The results show that to consistently and predictably improve retrieval effectiveness in the ideal SQE setting, a Kendall's Tau correlation of tau>=0.5 is required, a threshold which most existing query performance prediction methods fail to reach.
    Original languageUndefined
    Title of host publicationProceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
    Place of PublicationNew York
    PublisherAssociation for Computing Machinery (ACM)
    Pages830-831
    Number of pages2
    ISBN (Print)978-1-60558-483-6
    DOIs
    Publication statusPublished - 2009
    Event32nd Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009 - Boston, United States
    Duration: 19 Jul 200923 Jul 2009
    Conference number: 32

    Publication series

    Name
    PublisherACM

    Conference

    Conference32nd Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2009
    Abbreviated titleSIGIR
    CountryUnited States
    CityBoston
    Period19/07/0923/07/09

    Keywords

    • IR-67851
    • CR-H.3.3
    • EWI-15903
    • METIS-263976

    Cite this

    Hauff, C., & Azzopardi, L. (2009). When is query performance prediction effective? In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval (pp. 830-831). [10.1145/1571941.1572150] New York: Association for Computing Machinery (ACM). https://doi.org/10.1145/1571941.1572150