Flow monitoring explained: from packet capture to data analysis with NetFlow and IPFIX

R.J. Hofstede, Pavel Celeda, Brian Trammell, Idilio Drago, R. Sadre, Anna Sperotto, Aiko Pras

  • 79 Citations

Abstract

Flow monitoring has become a prevalent method for monitoring traffic in high-speed networks. By focusing on the analysis of flows, rather than individual packets, it is often said to be more scalable than traditional packet-based traffic analysis. Flow monitoring embraces the complete chain of packet observation, flow export using protocols such as NetFlow and IPFIX, data collection, and data analysis. In contrast to what is often assumed, all stages of flow monitoring are closely intertwined. Each of these stages therefore has to be thoroughly understood, before being able to perform sound flow measurements. Otherwise, flow data artifacts and data loss can be the consequence, potentially without being observed. This paper is the first of its kind to provide an integrated tutorial on all stages of a flow monitoring setup. As shown throughout this paper, flow monitoring has evolved from the early nineties into a powerful tool, and additional functionality will certainly be added in the future. We show, for example, how the previously opposing approaches of Deep Packet Inspection and flow monitoring have been united into novel monitoring approaches.
Original languageUndefined
Pages (from-to)2037-2064
Number of pages29
JournalIEEE communications surveys & tutorials
Volume16
Issue number4
DOIs
StatePublished - Nov 2014

Fingerprint

Monitoring
Acoustic variables measurement
HIgh speed networks
Flow measurement
Data reduction
Inspection

Keywords

  • EWI-25179
  • IR-93156
  • METIS-309610

Cite this

Hofstede, R.J.; Celeda, Pavel; Trammell, Brian; Drago, Idilio; Sadre, R.; Sperotto, Anna; Pras, Aiko / Flow monitoring explained: from packet capture to data analysis with NetFlow and IPFIX.

In: IEEE communications surveys & tutorials, Vol. 16, No. 4, 11.2014, p. 2037-2064.

Research output: Scientific - peer-reviewArticle

@article{016e2eda682a456c9bf162e1adbfb862,
title = "Flow monitoring explained: from packet capture to data analysis with NetFlow and IPFIX",
abstract = "Flow monitoring has become a prevalent method for monitoring traffic in high-speed networks. By focusing on the analysis of flows, rather than individual packets, it is often said to be more scalable than traditional packet-based traffic analysis. Flow monitoring embraces the complete chain of packet observation, flow export using protocols such as NetFlow and IPFIX, data collection, and data analysis. In contrast to what is often assumed, all stages of flow monitoring are closely intertwined. Each of these stages therefore has to be thoroughly understood, before being able to perform sound flow measurements. Otherwise, flow data artifacts and data loss can be the consequence, potentially without being observed. This paper is the first of its kind to provide an integrated tutorial on all stages of a flow monitoring setup. As shown throughout this paper, flow monitoring has evolved from the early nineties into a powerful tool, and additional functionality will certainly be added in the future. We show, for example, how the previously opposing approaches of Deep Packet Inspection and flow monitoring have been united into novel monitoring approaches.",
keywords = "EWI-25179, IR-93156, METIS-309610",
author = "R.J. Hofstede and Pavel Celeda and Brian Trammell and Idilio Drago and R. Sadre and Anna Sperotto and Aiko Pras",
note = "eemcs-eprint-25179",
year = "2014",
month = "11",
doi = "10.1109/COMST.2014.2321898",
volume = "16",
pages = "2037--2064",
journal = "IEEE communications surveys & tutorials",
issn = "1553-877X",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "4",

}

Flow monitoring explained: from packet capture to data analysis with NetFlow and IPFIX. / Hofstede, R.J.; Celeda, Pavel; Trammell, Brian; Drago, Idilio; Sadre, R.; Sperotto, Anna; Pras, Aiko.

In: IEEE communications surveys & tutorials, Vol. 16, No. 4, 11.2014, p. 2037-2064.

Research output: Scientific - peer-reviewArticle

TY - JOUR

T1 - Flow monitoring explained: from packet capture to data analysis with NetFlow and IPFIX

AU - Hofstede,R.J.

AU - Celeda,Pavel

AU - Trammell,Brian

AU - Drago,Idilio

AU - Sadre,R.

AU - Sperotto,Anna

AU - Pras,Aiko

N1 - eemcs-eprint-25179

PY - 2014/11

Y1 - 2014/11

N2 - Flow monitoring has become a prevalent method for monitoring traffic in high-speed networks. By focusing on the analysis of flows, rather than individual packets, it is often said to be more scalable than traditional packet-based traffic analysis. Flow monitoring embraces the complete chain of packet observation, flow export using protocols such as NetFlow and IPFIX, data collection, and data analysis. In contrast to what is often assumed, all stages of flow monitoring are closely intertwined. Each of these stages therefore has to be thoroughly understood, before being able to perform sound flow measurements. Otherwise, flow data artifacts and data loss can be the consequence, potentially without being observed. This paper is the first of its kind to provide an integrated tutorial on all stages of a flow monitoring setup. As shown throughout this paper, flow monitoring has evolved from the early nineties into a powerful tool, and additional functionality will certainly be added in the future. We show, for example, how the previously opposing approaches of Deep Packet Inspection and flow monitoring have been united into novel monitoring approaches.

AB - Flow monitoring has become a prevalent method for monitoring traffic in high-speed networks. By focusing on the analysis of flows, rather than individual packets, it is often said to be more scalable than traditional packet-based traffic analysis. Flow monitoring embraces the complete chain of packet observation, flow export using protocols such as NetFlow and IPFIX, data collection, and data analysis. In contrast to what is often assumed, all stages of flow monitoring are closely intertwined. Each of these stages therefore has to be thoroughly understood, before being able to perform sound flow measurements. Otherwise, flow data artifacts and data loss can be the consequence, potentially without being observed. This paper is the first of its kind to provide an integrated tutorial on all stages of a flow monitoring setup. As shown throughout this paper, flow monitoring has evolved from the early nineties into a powerful tool, and additional functionality will certainly be added in the future. We show, for example, how the previously opposing approaches of Deep Packet Inspection and flow monitoring have been united into novel monitoring approaches.

KW - EWI-25179

KW - IR-93156

KW - METIS-309610

U2 - 10.1109/COMST.2014.2321898

DO - 10.1109/COMST.2014.2321898

M3 - Article

VL - 16

SP - 2037

EP - 2064

JO - IEEE communications surveys & tutorials

T2 - IEEE communications surveys & tutorials

JF - IEEE communications surveys & tutorials

SN - 1553-877X

IS - 4

ER -

Hofstede RJ, Celeda P, Trammell B, Drago I, Sadre R, Sperotto A et al. Flow monitoring explained: from packet capture to data analysis with NetFlow and IPFIX. IEEE communications surveys & tutorials. 2014 Nov;16(4):2037-2064. Available from, DOI: 10.1109/COMST.2014.2321898