### Abstract

When the performance of a classifier is empirically evaluated, the Area Under Curve (AUC) is commonly used as a one dimensional performance measure. In general, the focus is on good performance (AUC towards 1). In this paper, we study the other side of the performance spectrum (AUC towards 0.50) as we are interested to which extend a classifier is random given its AUC. We present the exact probability distribution of the AUC of a truely random classifier, given a finite number of distinct genuine and imposter scores. It quantifies the randomness of the measured AUC. The distribution involves the restricted partition function, a well studied function in number theory. Although other work exists that considers confidence bounds on the AUC, the novelty is that we do not assume any underlying parametric or non- parametric model or specify an error rate. Also, in cases in which a limited number of scores is available, for example in forensic case work, the exact distribution can deviate from these models. For completeness, we also present an approximation using a normal distribution and confidence bounds on the AUC.

Original language | English |
---|---|

Title of host publication | 2017 International Conference of the Biometrics Special Interest Group (BIOSIG) |

Subtitle of host publication | BIOSIG 2017 |

Editors | Arslan Brömme, Christoph Busch, Antitza Dantcheva, Christian Rathgeb, Andreas Uhl |

Publisher | Gesellschaft für Informatik |

ISBN (Electronic) | 9783885796640 |

DOIs | |

Publication status | Published - 28 Sep 2017 |

Event | 16th International Conference of the Biometrics Special Interest Group 2017 - Darmstadt, Germany Duration: 20 Sep 2017 → 22 Sep 2017 Conference number: 16 http://fg-biosig.gi.de/archiv/biosig-2017.html |

### Conference

Conference | 16th International Conference of the Biometrics Special Interest Group 2017 |
---|---|

Abbreviated title | BIOSIG 2017 |

Country | Germany |

City | Darmstadt |

Period | 20/09/17 → 22/09/17 |

Internet address |

### Keywords

- Approximation.
- AUC
- Exact Distribution
- Random Classifier

## Fingerprint Dive into the research topics of 'How Random Is a Classifier Given Its Area under Curve?'. Together they form a unique fingerprint.

## Cite this

*2017 International Conference of the Biometrics Special Interest Group (BIOSIG): BIOSIG 2017*[8053509] Gesellschaft für Informatik. https://doi.org/10.23919/BIOSIG.2017.8053509