Evaluating and Predicting Actual Test Coverage

Research output: Book/ReportReportOther research output

3 Downloads (Pure)

Abstract

This thesis proposes a new notion of semantic coverage in formal testing: actual coverage. It is defined for test case and test suite executions, as well as for sequences of their executions. A fault is considered to be completely covered if an execution showed its presence, and it is considered partly covered if an execution increased the confidence in its absence.

Actual coverage can be used to evaluate a test process after it has taken place, but we also describe how to predict actual coverage in advance. To support these estimations, a probabilistic execution model is introduced. We derive efficient formulae for both the evaluation and the prediction of actual coverage, making tool support feasible.

We show that for an infinite number of executions our measure coincides with an existing notion of semantic coverage, called potential coverage. This notion, however, does not deal with the fact that in practice only a finite number of executions will be performed. With actual coverage it is possible to predict the actual coverage of any given number of test case or test suite executions.

An extensive detailed example is provided to demonstrate the applicability of our measure.
Original languageEnglish
Place of PublicationEnschede
PublisherUniversity of Twente
Number of pages133
Publication statusPublished - 24 Jun 2008

Fingerprint

Semantics
Testing
Statistical Models

Keywords

  • coverage
  • probabilistic
  • formal testing

Cite this

Timmer, M. (2008). Evaluating and Predicting Actual Test Coverage. Enschede: University of Twente.
Timmer, Mark . / Evaluating and Predicting Actual Test Coverage. Enschede : University of Twente, 2008. 133 p.
@book{c78122f3f0de40dc89f16c0f3b017861,
title = "Evaluating and Predicting Actual Test Coverage",
abstract = "This thesis proposes a new notion of semantic coverage in formal testing: actual coverage. It is defined for test case and test suite executions, as well as for sequences of their executions. A fault is considered to be completely covered if an execution showed its presence, and it is considered partly covered if an execution increased the confidence in its absence.Actual coverage can be used to evaluate a test process after it has taken place, but we also describe how to predict actual coverage in advance. To support these estimations, a probabilistic execution model is introduced. We derive efficient formulae for both the evaluation and the prediction of actual coverage, making tool support feasible.We show that for an infinite number of executions our measure coincides with an existing notion of semantic coverage, called potential coverage. This notion, however, does not deal with the fact that in practice only a finite number of executions will be performed. With actual coverage it is possible to predict the actual coverage of any given number of test case or test suite executions.An extensive detailed example is provided to demonstrate the applicability of our measure.",
keywords = "coverage, probabilistic, formal testing",
author = "Mark Timmer",
note = "MSc thesis",
year = "2008",
month = "6",
day = "24",
language = "English",
publisher = "University of Twente",
address = "Netherlands",

}

Timmer, M 2008, Evaluating and Predicting Actual Test Coverage. University of Twente, Enschede.

Evaluating and Predicting Actual Test Coverage. / Timmer, Mark .

Enschede : University of Twente, 2008. 133 p.

Research output: Book/ReportReportOther research output

TY - BOOK

T1 - Evaluating and Predicting Actual Test Coverage

AU - Timmer, Mark

N1 - MSc thesis

PY - 2008/6/24

Y1 - 2008/6/24

N2 - This thesis proposes a new notion of semantic coverage in formal testing: actual coverage. It is defined for test case and test suite executions, as well as for sequences of their executions. A fault is considered to be completely covered if an execution showed its presence, and it is considered partly covered if an execution increased the confidence in its absence.Actual coverage can be used to evaluate a test process after it has taken place, but we also describe how to predict actual coverage in advance. To support these estimations, a probabilistic execution model is introduced. We derive efficient formulae for both the evaluation and the prediction of actual coverage, making tool support feasible.We show that for an infinite number of executions our measure coincides with an existing notion of semantic coverage, called potential coverage. This notion, however, does not deal with the fact that in practice only a finite number of executions will be performed. With actual coverage it is possible to predict the actual coverage of any given number of test case or test suite executions.An extensive detailed example is provided to demonstrate the applicability of our measure.

AB - This thesis proposes a new notion of semantic coverage in formal testing: actual coverage. It is defined for test case and test suite executions, as well as for sequences of their executions. A fault is considered to be completely covered if an execution showed its presence, and it is considered partly covered if an execution increased the confidence in its absence.Actual coverage can be used to evaluate a test process after it has taken place, but we also describe how to predict actual coverage in advance. To support these estimations, a probabilistic execution model is introduced. We derive efficient formulae for both the evaluation and the prediction of actual coverage, making tool support feasible.We show that for an infinite number of executions our measure coincides with an existing notion of semantic coverage, called potential coverage. This notion, however, does not deal with the fact that in practice only a finite number of executions will be performed. With actual coverage it is possible to predict the actual coverage of any given number of test case or test suite executions.An extensive detailed example is provided to demonstrate the applicability of our measure.

KW - coverage

KW - probabilistic

KW - formal testing

M3 - Report

BT - Evaluating and Predicting Actual Test Coverage

PB - University of Twente

CY - Enschede

ER -

Timmer M. Evaluating and Predicting Actual Test Coverage. Enschede: University of Twente, 2008. 133 p.