Predicting the cost-quality trade-off for information retrieval queries: Facilitating database design and query optimization

H.E. Blok, Djoerd Hiemstra, R.S. Choenni, Franciska M.G. de Jong, Henk Blanken, Peter M.G. Apers

Research output: Book/ReportReportAcademic

27 Downloads (Pure)

Abstract

Efficient, exible, and scalable integration of full text information retrieval (IR) in a DBMS is not a trivial case. This holds in particular for query optimization in such a context. To facilitate the bulk-oriented behavior of database query processing, a priori knowledge of how to limit the data efficiently prior to query evaluation is very valuable at optimization time. The usually imprecise nature of IR querying provides an extra opportunity to limit the data by a trade-off with the quality of the answer. In this paper we present a mathematically derived model to predict the quality implications of neglecting information before query execution. In particular we investigate the possibility to predict the retrieval quality for a document collection for which no training information is available, which is usually the case in practice. Instead, we construct a model that can be trained on other document collections for which the necessary quality information is available, or can be obtained quite easily. We validate our model for several document collections and present the experimental results. These results show that our model performs quite well, even for the case were we did not train it on the test collection itself.
Original languageUndefined
PublisherCentre for Telematics and Information Technology (CTIT)
Number of pages10
Publication statusPublished - Sep 2001

Publication series

NameCTIT Technical Report Series
No.01-33
ISSN (Print)1381-3625

Keywords

  • METIS-202573
  • IR-36620
  • EWI-5902

Cite this

Blok, H. E., Hiemstra, D., Choenni, R. S., de Jong, F. M. G., Blanken, H., & Apers, P. M. G. (2001). Predicting the cost-quality trade-off for information retrieval queries: Facilitating database design and query optimization. (CTIT Technical Report Series; No. 01-33). Centre for Telematics and Information Technology (CTIT).