A High-Performance, Scalable Infrastructure for Large-Scale Active DNS Measurements

Roland M. van Rijswijk, Mattijs Jonker, Anna Sperotto, Aiko Pras

  • 9 Citations

Abstract

The domain name system (DNS) is a core component of the Internet. It performs the vital task of mapping human readable names into machine readable data (such as IP addresses, which hosts handle e-mail, and so on). The content of the DNS reveals a lot about the technical operations of a domain. Thus, studying the state of large parts of the DNS over time reveals valuable information about the evolution of the Internet. We collect a unique long-term data set with daily DNS measurements for all the domains under the main top-level domains (TLDs) on the Internet (including .com, .net, and .org, comprising 50% of the global DNS name space). This paper discusses the challenges of performing such a large-scale active measurement. These challenges include scaling the daily measurement to collect data for the largest TLD (.com, with 123M names) and ensuring that a measurement of this scale does not impose an unacceptable burden on the global DNS infrastructure. The paper discusses the design choices we have made to meet these challenges and documents the design of the measurement system we implemented based on these choices. Two case studies related to cloud e-mail services illustrate the value of measuring the DNS at this scale. The data this system collects is valuable to the network research community. Therefore, we end this paper by discussing how we make the data accessible to other researchers.
Original languageUndefined
Pages (from-to)1877-1888
Number of pages12
JournalIEEE journal on selected areas in communications
Volume34
Issue number6
DOIs
StatePublished - Jun 2016

Fingerprint

Internet

Keywords

  • EWI-27652
  • active measurements
  • IR-104117
  • DNS
  • Internet evolution
  • cloud

Cite this

van Rijswijk, Roland M.; Jonker, Mattijs; Sperotto, Anna; Pras, Aiko / A High-Performance, Scalable Infrastructure for Large-Scale Active DNS Measurements.

In: IEEE journal on selected areas in communications, Vol. 34, No. 6, 06.2016, p. 1877-1888.

Research output: Scientific - peer-reviewArticle

@article{865063ba7c9d40e1aa172e9c74d9dfe7,
title = "A High-Performance, Scalable Infrastructure for Large-Scale Active DNS Measurements",
abstract = "The domain name system (DNS) is a core component of the Internet. It performs the vital task of mapping human readable names into machine readable data (such as IP addresses, which hosts handle e-mail, and so on). The content of the DNS reveals a lot about the technical operations of a domain. Thus, studying the state of large parts of the DNS over time reveals valuable information about the evolution of the Internet. We collect a unique long-term data set with daily DNS measurements for all the domains under the main top-level domains (TLDs) on the Internet (including .com, .net, and .org, comprising 50% of the global DNS name space). This paper discusses the challenges of performing such a large-scale active measurement. These challenges include scaling the daily measurement to collect data for the largest TLD (.com, with 123M names) and ensuring that a measurement of this scale does not impose an unacceptable burden on the global DNS infrastructure. The paper discusses the design choices we have made to meet these challenges and documents the design of the measurement system we implemented based on these choices. Two case studies related to cloud e-mail services illustrate the value of measuring the DNS at this scale. The data this system collects is valuable to the network research community. Therefore, we end this paper by discussing how we make the data accessible to other researchers.",
keywords = "EWI-27652, active measurements, IR-104117, DNS, Internet evolution, cloud",
author = "{van Rijswijk}, {Roland M.} and Mattijs Jonker and Anna Sperotto and Aiko Pras",
year = "2016",
month = "6",
doi = "10.1109/JSAC.2016.2558918",
volume = "34",
pages = "1877--1888",
journal = "IEEE journal on selected areas in communications",
issn = "0733-8716",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "6",

}

A High-Performance, Scalable Infrastructure for Large-Scale Active DNS Measurements. / van Rijswijk, Roland M.; Jonker, Mattijs; Sperotto, Anna; Pras, Aiko.

In: IEEE journal on selected areas in communications, Vol. 34, No. 6, 06.2016, p. 1877-1888.

Research output: Scientific - peer-reviewArticle

TY - JOUR

T1 - A High-Performance, Scalable Infrastructure for Large-Scale Active DNS Measurements

AU - van Rijswijk,Roland M.

AU - Jonker,Mattijs

AU - Sperotto,Anna

AU - Pras,Aiko

PY - 2016/6

Y1 - 2016/6

N2 - The domain name system (DNS) is a core component of the Internet. It performs the vital task of mapping human readable names into machine readable data (such as IP addresses, which hosts handle e-mail, and so on). The content of the DNS reveals a lot about the technical operations of a domain. Thus, studying the state of large parts of the DNS over time reveals valuable information about the evolution of the Internet. We collect a unique long-term data set with daily DNS measurements for all the domains under the main top-level domains (TLDs) on the Internet (including .com, .net, and .org, comprising 50% of the global DNS name space). This paper discusses the challenges of performing such a large-scale active measurement. These challenges include scaling the daily measurement to collect data for the largest TLD (.com, with 123M names) and ensuring that a measurement of this scale does not impose an unacceptable burden on the global DNS infrastructure. The paper discusses the design choices we have made to meet these challenges and documents the design of the measurement system we implemented based on these choices. Two case studies related to cloud e-mail services illustrate the value of measuring the DNS at this scale. The data this system collects is valuable to the network research community. Therefore, we end this paper by discussing how we make the data accessible to other researchers.

AB - The domain name system (DNS) is a core component of the Internet. It performs the vital task of mapping human readable names into machine readable data (such as IP addresses, which hosts handle e-mail, and so on). The content of the DNS reveals a lot about the technical operations of a domain. Thus, studying the state of large parts of the DNS over time reveals valuable information about the evolution of the Internet. We collect a unique long-term data set with daily DNS measurements for all the domains under the main top-level domains (TLDs) on the Internet (including .com, .net, and .org, comprising 50% of the global DNS name space). This paper discusses the challenges of performing such a large-scale active measurement. These challenges include scaling the daily measurement to collect data for the largest TLD (.com, with 123M names) and ensuring that a measurement of this scale does not impose an unacceptable burden on the global DNS infrastructure. The paper discusses the design choices we have made to meet these challenges and documents the design of the measurement system we implemented based on these choices. Two case studies related to cloud e-mail services illustrate the value of measuring the DNS at this scale. The data this system collects is valuable to the network research community. Therefore, we end this paper by discussing how we make the data accessible to other researchers.

KW - EWI-27652

KW - active measurements

KW - IR-104117

KW - DNS

KW - Internet evolution

KW - cloud

U2 - 10.1109/JSAC.2016.2558918

DO - 10.1109/JSAC.2016.2558918

M3 - Article

VL - 34

SP - 1877

EP - 1888

JO - IEEE journal on selected areas in communications

T2 - IEEE journal on selected areas in communications

JF - IEEE journal on selected areas in communications

SN - 0733-8716

IS - 6

ER -