Query quality: user ratings and system predictions

C. Hauff, Franciska M.G. de Jong, Diane Kelly, Leif Azzopardi

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    5 Citations (Scopus)
    19 Downloads (Pure)

    Abstract

    Numerous studies have examined the ability of query performance prediction methods to estimate a query's quality for system effectiveness measures (such as average precision). However, little work has explored the relationship between these methods and user ratings of query quality. In this poster, we report the findings from an empirical study conducted on the TREC ClueWeb09 corpus, where we compared and contrasted user ratings of query quality against a range of query performance prediction methods. Given a set of queries, it is shown that user ratings of query quality correlate to both system effectiveness measures and a number of pre-retrieval predictors.
    Original languageUndefined
    Title of host publicationProceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval
    Place of PublicationNew York
    PublisherAssociation for Computing Machinery (ACM)
    Pages743-744
    Number of pages2
    ISBN (Print)978-1-4503-0153-4
    DOIs
    Publication statusPublished - Jul 2010
    Event33rd Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010 - Geneva, Switzerland
    Duration: 19 Jul 201023 Jul 2010
    Conference number: 33

    Publication series

    Name
    PublisherACM

    Conference

    Conference33rd Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010
    Abbreviated titleSIGIR
    CountrySwitzerland
    CityGeneva
    Period19/07/1023/07/10

    Keywords

    • METIS-270948
    • IR-72486
    • CR-H.3
    • user ratings of query quality
    • EWI-18229
    • Query performance prediction

    Cite this