Quantitative penetration testing with item response theory (extended version)

    Research output: Book/ReportReportProfessional

    73 Downloads (Pure)

    Abstract

    Existing penetration testing approaches assess the vulnerability of a system by determining whether certain attack paths are possible in practice. Therefore, penetration testing has thus far been used as a qualitative research method. To enable quantitative approaches to security risk management, including decision support based on the cost-effectiveness of countermeasures, one needs quantitative measures of the feasibility of an attack. Also, when physical or social attack steps are involved, the binary view on whether a vulnerability is present or not is insucient, and one needs some viability metric. When penetration tests are performed anyway, it is very easy for the testers to keep track of, for example, the time they spend on each attack step. Therefore, this paper proposes the concept of quantitative penetration testing to determine the diculty rather than the possibility of attacks. We do this by step-wise updates of expected time and probability of success for all steps in an attack scenario. Also, the skill of the testers can be included to improve the accuracy of the metrics, based on the framework of Item Response Theory (Elo ratings). We show the feasibility of the approach by means of simulations, and discuss application possibilities.
    Original languageUndefined
    Place of PublicationEnschede
    PublisherCentre for Telematics and Information Technology (CTIT)
    Number of pages11
    Publication statusPublished - Oct 2013

    Publication series

    NameCTIT Technical Report Series
    PublisherUniversity of Twente, Centre for Telematics and Information Technology (CTIT)
    No.TR-CTIT-13-20
    ISSN (Print)1381-3625

    Keywords

    • EC Grant Agreement nr.: FP7/2007-2013
    • Security Metrics
    • EWI-23863
    • IR-87631
    • METIS-300106
    • Socio-technical security
    • SCS-Cybersecurity
    • Item Response Theory
    • Penetration Testing
    • quantitative security
    • EC Grant Agreement nr.: FP7/318003

    Cite this

    Arnold, F., Pieters, W., & Stoelinga, M. I. A. (2013). Quantitative penetration testing with item response theory (extended version). (CTIT Technical Report Series; No. TR-CTIT-13-20). Enschede: Centre for Telematics and Information Technology (CTIT).
    Arnold, Florian ; Pieters, Wolter ; Stoelinga, Mariëlle Ida Antoinette. / Quantitative penetration testing with item response theory (extended version). Enschede : Centre for Telematics and Information Technology (CTIT), 2013. 11 p. (CTIT Technical Report Series; TR-CTIT-13-20).
    @book{1356bd7b609f4a888d4a2f3059ba3704,
    title = "Quantitative penetration testing with item response theory (extended version)",
    abstract = "Existing penetration testing approaches assess the vulnerability of a system by determining whether certain attack paths are possible in practice. Therefore, penetration testing has thus far been used as a qualitative research method. To enable quantitative approaches to security risk management, including decision support based on the cost-effectiveness of countermeasures, one needs quantitative measures of the feasibility of an attack. Also, when physical or social attack steps are involved, the binary view on whether a vulnerability is present or not is insucient, and one needs some viability metric. When penetration tests are performed anyway, it is very easy for the testers to keep track of, for example, the time they spend on each attack step. Therefore, this paper proposes the concept of quantitative penetration testing to determine the diculty rather than the possibility of attacks. We do this by step-wise updates of expected time and probability of success for all steps in an attack scenario. Also, the skill of the testers can be included to improve the accuracy of the metrics, based on the framework of Item Response Theory (Elo ratings). We show the feasibility of the approach by means of simulations, and discuss application possibilities.",
    keywords = "EC Grant Agreement nr.: FP7/2007-2013, Security Metrics, EWI-23863, IR-87631, METIS-300106, Socio-technical security, SCS-Cybersecurity, Item Response Theory, Penetration Testing, quantitative security, EC Grant Agreement nr.: FP7/318003",
    author = "Florian Arnold and Wolter Pieters and Stoelinga, {Mari{\"e}lle Ida Antoinette}",
    note = "Foreground = 100{\%}; Type of activity = technical report; Main leader = UT; Type of audience = scientific community; Size of audience = n.a.; Countries addressed = international;",
    year = "2013",
    month = "10",
    language = "Undefined",
    series = "CTIT Technical Report Series",
    publisher = "Centre for Telematics and Information Technology (CTIT)",
    number = "TR-CTIT-13-20",
    address = "Netherlands",

    }

    Arnold, F, Pieters, W & Stoelinga, MIA 2013, Quantitative penetration testing with item response theory (extended version). CTIT Technical Report Series, no. TR-CTIT-13-20, Centre for Telematics and Information Technology (CTIT), Enschede.

    Quantitative penetration testing with item response theory (extended version). / Arnold, Florian; Pieters, Wolter; Stoelinga, Mariëlle Ida Antoinette.

    Enschede : Centre for Telematics and Information Technology (CTIT), 2013. 11 p. (CTIT Technical Report Series; No. TR-CTIT-13-20).

    Research output: Book/ReportReportProfessional

    TY - BOOK

    T1 - Quantitative penetration testing with item response theory (extended version)

    AU - Arnold, Florian

    AU - Pieters, Wolter

    AU - Stoelinga, Mariëlle Ida Antoinette

    N1 - Foreground = 100%; Type of activity = technical report; Main leader = UT; Type of audience = scientific community; Size of audience = n.a.; Countries addressed = international;

    PY - 2013/10

    Y1 - 2013/10

    N2 - Existing penetration testing approaches assess the vulnerability of a system by determining whether certain attack paths are possible in practice. Therefore, penetration testing has thus far been used as a qualitative research method. To enable quantitative approaches to security risk management, including decision support based on the cost-effectiveness of countermeasures, one needs quantitative measures of the feasibility of an attack. Also, when physical or social attack steps are involved, the binary view on whether a vulnerability is present or not is insucient, and one needs some viability metric. When penetration tests are performed anyway, it is very easy for the testers to keep track of, for example, the time they spend on each attack step. Therefore, this paper proposes the concept of quantitative penetration testing to determine the diculty rather than the possibility of attacks. We do this by step-wise updates of expected time and probability of success for all steps in an attack scenario. Also, the skill of the testers can be included to improve the accuracy of the metrics, based on the framework of Item Response Theory (Elo ratings). We show the feasibility of the approach by means of simulations, and discuss application possibilities.

    AB - Existing penetration testing approaches assess the vulnerability of a system by determining whether certain attack paths are possible in practice. Therefore, penetration testing has thus far been used as a qualitative research method. To enable quantitative approaches to security risk management, including decision support based on the cost-effectiveness of countermeasures, one needs quantitative measures of the feasibility of an attack. Also, when physical or social attack steps are involved, the binary view on whether a vulnerability is present or not is insucient, and one needs some viability metric. When penetration tests are performed anyway, it is very easy for the testers to keep track of, for example, the time they spend on each attack step. Therefore, this paper proposes the concept of quantitative penetration testing to determine the diculty rather than the possibility of attacks. We do this by step-wise updates of expected time and probability of success for all steps in an attack scenario. Also, the skill of the testers can be included to improve the accuracy of the metrics, based on the framework of Item Response Theory (Elo ratings). We show the feasibility of the approach by means of simulations, and discuss application possibilities.

    KW - EC Grant Agreement nr.: FP7/2007-2013

    KW - Security Metrics

    KW - EWI-23863

    KW - IR-87631

    KW - METIS-300106

    KW - Socio-technical security

    KW - SCS-Cybersecurity

    KW - Item Response Theory

    KW - Penetration Testing

    KW - quantitative security

    KW - EC Grant Agreement nr.: FP7/318003

    M3 - Report

    T3 - CTIT Technical Report Series

    BT - Quantitative penetration testing with item response theory (extended version)

    PB - Centre for Telematics and Information Technology (CTIT)

    CY - Enschede

    ER -

    Arnold F, Pieters W, Stoelinga MIA. Quantitative penetration testing with item response theory (extended version). Enschede: Centre for Telematics and Information Technology (CTIT), 2013. 11 p. (CTIT Technical Report Series; TR-CTIT-13-20).