Safe Testing

Peter Grünwald*, Rianne de Heide, Wouter Koolen

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

30 Downloads (Pure)

Abstract

We develop the theory of hypothesis testing based on the e-value, a notion of evidence that, unlike the p-value, allows for effortlessly combining results from several studies in the common scenario where the decision to perform a new study may depend on previous outcomes. Tests based on e-values are safe, i.e. they preserve Type-I error guarantees, under such optional continuation. We define growthrate optimality (GRO) as an analogue of power in an optional continuation context, and we show how to construct GRO e-variables for general testing problems with composite null and alternative, emphasizing models with nuisance parameters. GRO e-values take the form of Bayes factors with special priors. We illustrate the theory using several classic examples including a one-sample safe t -test and the 2x2 contingency table. Sharing Fisherian, Neymanian and Jeffreys-Bayesian interpretations, e-values may provide a methodology acceptable to adherents of all three schools.
Original languageEnglish
Number of pages38
JournalJournal of the Royal Statistical Society. Series B: Statistical Methodology
Early online date7 Mar 2024
DOIs
Publication statusE-pub ahead of print/First online - 7 Mar 2024

Fingerprint

Dive into the research topics of 'Safe Testing'. Together they form a unique fingerprint.
  • Safe Testing

    Grünwald, P., de Heide, R. & Koolen, W. M., 18 Jun 2019, ArXiv.org.

    Research output: Working paperPreprintAcademic

    File
    12 Downloads (Pure)

Cite this