Flow-Based Intrusion Detection

The spread of 1-10 Gbps technology has in recent years paved the way to a flourishing landscape of new, high-bandwidth Internet services.At the same time, we have also observed increasingly frequent and widely diversified attacks. To this threat, the research community has answered with a growing interest in intrusion detection, aiming to timely detect intruders and prevent damage. We believe that the detection problem is a key component in the field of intrusion detection. Our studies, however, made us realize that additional research is needed, in particular focusing on validation and automatic tuning of Intrusion Detection Systems (IDSs). The contribution of this thesis is that it develops a structured approach to intrusion detection that focuses on (i) system validation and (ii) automatic system tuning. We developed our approach by focusing on network flows, which offer an aggregated view of network traffic and help to cope with scalability issues. An interesting approach to validation is the creation of appropriate testbeds, or ground-truth data sets, for which it is known when an attack has taken place. First, we obtained ground-truth information for flow-based intrusion detection by manually creating it. The outcome of our research is the first publicly released flow-based labeled data set. Second, we generated ground truth information in an automatic manner, by means of probabilistic traffic models based on Hidden Markov Models (HMMs). Finally, we approached the problem of automatic tuning of IDSs. The performance of an IDS is governed by the trade-off between detecting all anomalies (at the expense of raising alarms too often), and missing anomalies (but not issuing many false alarms). We developed an optimization procedure that aims to mathematically treat such trade-off in a systematic manner, by automatically tuning the system parameters.