Monitoring User-System Performance in Interactive Retrieval Tasks

Liudmila Boldareva, Arjen P. de Vries, Djoerd Hiemstra

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Monitoring user-system performance in interactive search is a challenging task. Traditional measures of retrieval evaluation, based on recall and precision, are not of any use in real time, for they require a priori knowledge of relevant documents. This paper shows how a Shannon entropy-based measure of user-system performance naturally falls in the framework of (interactive) probabilistic information retrieval. The value of entropy of the distribution of probability of relevance associated with the documents in the collection can be used to monitor search progress in live testing, to allow for example the system to select an optimal combination of search strategies. User profiling and tuning parameters of retrieval systems are other important applications.
Original languageEnglish
Title of host publicationRIAO 2004
Subtitle of host publicationconference proceedings Coupling approaches, coupling media and coupling languages for information retrieval, University of Avignon (Vaucluse), France, April 26-28, 2004
Place of PublicationParis
PublisherCentre de Hautes Etudes Internationales d'Informatique Documentaire (CID)
Pages474-483
Number of pages10
ISBN (Print)9782905450098
Publication statusPublished - Apr 2004
Event7th International Conference on Computer-Assisted Information Retrieval, RIAO 2004: (Recherche d'Information et ses Applications) - Avignon, France
Duration: 1 Apr 20041 Apr 2004
Conference number: 7

Conference

Conference7th International Conference on Computer-Assisted Information Retrieval, RIAO 2004
Abbreviated titleRIAO
CountryFrance
CityAvignon
Period1/04/041/04/04

Keywords

  • DB-IR: INFORMATION RETRIEVAL
  • EWI-7176
  • IR-63495
  • METIS-221593

Fingerprint Dive into the research topics of 'Monitoring User-System Performance in Interactive Retrieval Tasks'. Together they form a unique fingerprint.

Cite this