Federated Search in the Wild: the combined power of over a hundred search engines

Dong-Phuong Nguyen, Thomas Demeester, Rudolf Berend Trieschnigg, Djoerd Hiemstra

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

31 Citations (Scopus)
67 Downloads (Pure)


Federated search has the potential of improving web search: the user becomes less dependent on a single search provider and parts of the deep web become available through a unified interface, leading to a wider variety in the retrieved search results. However, a publicly available dataset for federated search reflecting an actual web environment has been absent. As a result, it has been difficult to assess whether proposed systems are suitable for the web setting. We introduce a new test collection containing the results from more than a hundred actual search engines, ranging from large general web search engines such as Google and Bing to small domain-specific engines. We discuss the design and analyze the effect of several sampling methods. For a set of test queries, we collected relevance judgements for the top 10 results of each search engine. The dataset is publicly available and is useful for researchers interested in resource selection for web search collections, result merging and size estimation of uncooperative resources.
Original languageUndefined
Title of host publicationProceedings of the 21th ACM international Conference on Information and Knowledge Management (CIKM 2012)
Place of PublicationNew York
PublisherAssociation for Computing Machinery
Number of pages5
ISBN (Print)978-1-4503-1156-4
Publication statusPublished - Oct 2012
Event21th ACM international Conference on Information and Knowledge Management, CIKM 2012 - Maui, United States
Duration: 29 Oct 20122 Nov 2012
Conference number: 21

Publication series



Conference21th ACM international Conference on Information and Knowledge Management, CIKM 2012
Abbreviated titleCIKM
Country/TerritoryUnited States


  • test collection
  • dataset
  • web search
  • EWI-22480
  • METIS-296137
  • Federated search
  • IR-83405
  • Evaluation
  • Distributed Information Retrieval

Cite this