This report deals with the testing of eSTV software from ERS, Electoral Reform Services. The
purpose of the test is to determine the conformance of eSTV to the WIG-rule . In order
to determine conformance we have created a new implementation of the rule in the functional
programming language Clean . Since this language uses a different programming paradigm
and the implementation is made completely independent from the software to be tested it is
extremely unlikely that both implementations contain the same error.
Testing is done by computing the election result for a given set of votes with both programs.
The results of both programs are compared to the level of votes numbers after each selection
and elimination step in the algorithm. This has shown to be a very sensitive test for very small
variations in the algorithm used in the programs.
These tests are executed for 808 standard test cases. These test cases are partly handcrafted
or the vote distributions in real elections for similar voting rules. Each of these sets is computed
with 4 different elimination orders. In addition the results of 5000 generated data sets and a
number of hand crafted data sets covering border situations in the algorithm are compared.
Both test sets have shown to be very effective to identify even the smallest changes in the
algorithm to compute the result of the election.
Testing of preliminary versions of eSTV showed some small inaccuracies in the program
as well as some small space for different interpretation of the rules. The Scottish Executive
provided the final interpretation of the rules (as detailed in section 7.1) and all test where
repeated for eSTV version 2.0.16. In these final tests there was an exact match for each of the
4 × 808 + 5000 = 8232 test cases.
|Publisher||Radboud University Nijmegen|