Statistical Language Models and Information Retrieval: Natural Language Processing Really Meets Retrieval

Traditionally, natural language processing techniques for information retrieval have always been studied outside the framework of formal models of information retrieval. In this article, we introduce a new formal model of information retrieval based on the application of statistical language models. Simple natural language processing techniques that are often used for information retrieval ± we give an introductory overview of these techniques in Section 2 ± can be modeled by the new language modeling approach.