Actual Test Coverage for Embedded Systems

Testing embedded systems is inherently incomplete; no test suite will ever be able to test all possible usage scenarios. Therefore, in the past decades many coverage measures have been developed. These measures denote the portion of a system that is tested, that way providing a quality criterion for test suites. Formulating coverage criteria is not an easy task. The measures provided in the literature are consequently almost all very trivial and syntax-dependent. Well-known examples are statement and path coverage in white-box testing, and state and transition coverage in black-box testing. The complexity of designing coverage measures for embedded systems is contained in the highly dynamic behaviour of such systems, which is state-dependent and subject to many interleavings. In this talk we introduce a framework on actual test coverage. This measure denotes the number of faults actually shown present or absent. Our framework contains a method to evaluate the actual coverage of a given set of test suite executions after testing has taken place, providing a means to express the quality of a testing process. It also contains a method to predict the actual coverage a certain number of executions will yield, providing a means to select the best test suite. Both the evaluation afterwards and the prediction in advance are quite efficient, making it feasible to implement the theory in a tool and use it in a practical context.