Query quality: user ratings and system predictions

C. Hauff; Franciska M.G. de Jong; Diane Kelly; Leif Azzopardi

doi:10.1145/1835449.1835594

Query quality: user ratings and system predictions

C. Hauff
, Franciska M.G. de Jong
, Diane Kelly
, Leif Azzopardi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

5 Citations (Scopus)

95 Downloads (Pure)

Abstract

Numerous studies have examined the ability of query performance prediction methods to estimate a query's quality for system effectiveness measures (such as average precision). However, little work has explored the relationship between these methods and user ratings of query quality. In this poster, we report the findings from an empirical study conducted on the TREC ClueWeb09 corpus, where we compared and contrasted user ratings of query quality against a range of query performance prediction methods. Given a set of queries, it is shown that user ratings of query quality correlate to both system effectiveness measures and a number of pre-retrieval predictors.

Original language	Undefined
Title of host publication	Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval
Place of Publication	New York
Publisher	Association for Computing Machinery
Pages	743-744
Number of pages	2
ISBN (Print)	978-1-4503-0153-4
DOIs	https://doi.org/10.1145/1835449.1835594
Publication status	Published - Jul 2010
Event	33rd Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010 - Geneva, Switzerland Duration: 19 Jul 2010 → 23 Jul 2010 Conference number: 33

Publication series

Name
Publisher	ACM

Conference

Conference	33rd Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010
Abbreviated title	SIGIR
Country/Territory	Switzerland
City	Geneva
Period	19/07/10 → 23/07/10

Keywords

METIS-270948
IR-72486
CR-H.3
user ratings of query quality
EWI-18229
Query performance prediction

Access to Document

10.1145/1835449.1835594

p743-hauff

Cite this

@inproceedings{4735651632d546abb9f49b2fa7ed71f6,

title = "Query quality: user ratings and system predictions",

abstract = "Numerous studies have examined the ability of query performance prediction methods to estimate a query's quality for system effectiveness measures (such as average precision). However, little work has explored the relationship between these methods and user ratings of query quality. In this poster, we report the findings from an empirical study conducted on the TREC ClueWeb09 corpus, where we compared and contrasted user ratings of query quality against a range of query performance prediction methods. Given a set of queries, it is shown that user ratings of query quality correlate to both system effectiveness measures and a number of pre-retrieval predictors.",

keywords = "METIS-270948, IR-72486, CR-H.3, user ratings of query quality, EWI-18229, Query performance prediction",

author = "C. Hauff and \{de Jong\}, \{Franciska M.G.\} and Diane Kelly and Leif Azzopardi",

note = "10.1145/1835449.1835594 ; 33rd Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010 ; Conference date: 19-07-2010 Through 23-07-2010",

year = "2010",

month = jul,

doi = "10.1145/1835449.1835594",

language = "Undefined",

isbn = "978-1-4503-0153-4",

publisher = "Association for Computing Machinery",

pages = "743--744",

booktitle = "Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval",

address = "United States",

}

Hauff, C, de Jong, FMG, Kelly, D & Azzopardi, L 2010, Query quality: user ratings and system predictions. in Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval., 10.1145/1835449.1835594, Association for Computing Machinery, New York, pp. 743-744, 33rd Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010, Geneva, Switzerland, 19/07/10. https://doi.org/10.1145/1835449.1835594

Query quality: user ratings and system predictions. / Hauff, C.; de Jong, Franciska M.G.; Kelly, Diane et al.
Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval. New York: Association for Computing Machinery, 2010. p. 743-744 10.1145/1835449.1835594.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Query quality: user ratings and system predictions

AU - Hauff, C.

AU - de Jong, Franciska M.G.

AU - Kelly, Diane

AU - Azzopardi, Leif

N1 - Conference code: 33

PY - 2010/7

Y1 - 2010/7

N2 - Numerous studies have examined the ability of query performance prediction methods to estimate a query's quality for system effectiveness measures (such as average precision). However, little work has explored the relationship between these methods and user ratings of query quality. In this poster, we report the findings from an empirical study conducted on the TREC ClueWeb09 corpus, where we compared and contrasted user ratings of query quality against a range of query performance prediction methods. Given a set of queries, it is shown that user ratings of query quality correlate to both system effectiveness measures and a number of pre-retrieval predictors.

AB - Numerous studies have examined the ability of query performance prediction methods to estimate a query's quality for system effectiveness measures (such as average precision). However, little work has explored the relationship between these methods and user ratings of query quality. In this poster, we report the findings from an empirical study conducted on the TREC ClueWeb09 corpus, where we compared and contrasted user ratings of query quality against a range of query performance prediction methods. Given a set of queries, it is shown that user ratings of query quality correlate to both system effectiveness measures and a number of pre-retrieval predictors.

KW - METIS-270948

KW - IR-72486

KW - CR-H.3

KW - user ratings of query quality

KW - EWI-18229

KW - Query performance prediction

U2 - 10.1145/1835449.1835594

DO - 10.1145/1835449.1835594

M3 - Conference contribution

SN - 978-1-4503-0153-4

SP - 743

EP - 744

BT - Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval

PB - Association for Computing Machinery

CY - New York

T2 - 33rd Annual International ACM/SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010

Y2 - 19 July 2010 through 23 July 2010

ER -