Abstract
Efficient, flexible, and scalable integration of full text information retrieval (IR) in a DBMS is not a trivial case. This holds in particular for query optimization in such a context. To facilitate the bulk-oriented behavior of database query processing, a priori knowledge of how to limit the data efficiently prior to query evaluation is very valuable at optimization time. The usually imprecise nature of IR querying provides an extra opportunity to limit the data by a trade-off with the quality of the answer. In this paper we present a mathematically derived model to predict the quality implications of neglecting information before query execution. In particular we investigate the possibility to predict the retrieval quality for a document collection for which no training information is available, which is usually the case in practice. Instead, we construct a model that can be trained on other document collections for which the necessary quality information is available, or can be obtained quite easily. We validate our model for several document collections and present the experimental results. These results show that our model performs quite well, even for the case were we did not train it on the test collection itself.
Original language | English |
---|---|
Title of host publication | CIKM '01 |
Subtitle of host publication | Proceedings of the Tenth International Conference on Information and Knowledge Management |
Place of Publication | New York, NY, USA |
Publisher | ACM Press |
Pages | 207-214 |
Number of pages | 8 |
ISBN (Print) | 1-58113-436-3 |
DOIs | |
Publication status | Published - Nov 2001 |
Event | 10th International Conference on Information and Knowledge Management, CIKM 2001 - Atlanta, United States Duration: 5 Nov 2001 → 10 Nov 2001 Conference number: 10 |
Conference
Conference | 10th International Conference on Information and Knowledge Management, CIKM 2001 |
---|---|
Abbreviated title | CIKM |
Country/Territory | United States |
City | Atlanta |
Period | 5/11/01 → 10/11/01 |
Keywords
- DB-IR: INFORMATION RETRIEVAL
- Quality
- Efficiency
- Trade-offs
- Fragmentation
- Zipf
- Information retrieval
- Databases