Bayesian variance estimation in the Gaussian sequence model with partial information on the means

Gianluca Finocchio*, Anselm Johannes Schmidt-Hieber*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

9 Downloads (Pure)

Abstract

Consider the Gaussian sequence model under the additional assumption that a fixed fraction of the means is known. We study the problem of variance estimation from a frequentist Bayesian perspective. The maximum likelihood estimator (MLE) for σ2 is biased and inconsistent. This raises the question whether the posterior is able to correct the MLE in this case. By developing a new proving strategy that uses refined properties of the posterior distribution, we find that the marginal posterior is inconsistent for any i.i.d. prior on the mean parameters. In particular, no assumption on the decay of the prior needs to be imposed. Surprisingly, we also find that consistency can be retained for a hierarchical prior based on Gaussian mixtures. In this case we also establish a limiting shape result and determine the limit distribution. In contrast to the classical Bernstein-von Mises theorem, the limit is non-Gaussian. We show that the Bayesian analysis leads to new statistical estimators outperforming the correctly calibrated MLE in a numerical simulation study.
Original languageEnglish
Pages (from-to)239-271
JournalElectronic Journal of Statistics
Volume14
Issue number1
DOIs
Publication statusPublished - 8 Jan 2020

Fingerprint

Variance Estimation
Partial Information
Bayesian Estimation
Maximum Likelihood Estimator
Inconsistent
Hierarchical Prior
Gaussian Mixture
Limit Distribution
Bayesian Analysis
Posterior distribution
Biased
Limiting
Simulation Study
Model
Decay
Estimator
Numerical Simulation
Theorem
Variance estimation
Maximum likelihood estimator

Cite this

Finocchio, Gianluca ; Schmidt-Hieber, Anselm Johannes. / Bayesian variance estimation in the Gaussian sequence model with partial information on the means. In: Electronic Journal of Statistics. 2020 ; Vol. 14, No. 1. pp. 239-271.
@article{cbca36ae3a1f4877a5cc4fa60f2d55a5,
title = "Bayesian variance estimation in the Gaussian sequence model with partial information on the means",
abstract = "Consider the Gaussian sequence model under the additional assumption that a fixed fraction of the means is known. We study the problem of variance estimation from a frequentist Bayesian perspective. The maximum likelihood estimator (MLE) for σ2 is biased and inconsistent. This raises the question whether the posterior is able to correct the MLE in this case. By developing a new proving strategy that uses refined properties of the posterior distribution, we find that the marginal posterior is inconsistent for any i.i.d. prior on the mean parameters. In particular, no assumption on the decay of the prior needs to be imposed. Surprisingly, we also find that consistency can be retained for a hierarchical prior based on Gaussian mixtures. In this case we also establish a limiting shape result and determine the limit distribution. In contrast to the classical Bernstein-von Mises theorem, the limit is non-Gaussian. We show that the Bayesian analysis leads to new statistical estimators outperforming the correctly calibrated MLE in a numerical simulation study.",
author = "Gianluca Finocchio and Schmidt-Hieber, {Anselm Johannes}",
year = "2020",
month = "1",
day = "8",
doi = "10.1214/19-EJS1671",
language = "English",
volume = "14",
pages = "239--271",
journal = "Electronic Journal of Statistics",
issn = "1935-7524",
publisher = "Institute of Mathematical Statistics",
number = "1",

}

Bayesian variance estimation in the Gaussian sequence model with partial information on the means. / Finocchio, Gianluca ; Schmidt-Hieber, Anselm Johannes.

In: Electronic Journal of Statistics, Vol. 14, No. 1, 08.01.2020, p. 239-271.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Bayesian variance estimation in the Gaussian sequence model with partial information on the means

AU - Finocchio, Gianluca

AU - Schmidt-Hieber, Anselm Johannes

PY - 2020/1/8

Y1 - 2020/1/8

N2 - Consider the Gaussian sequence model under the additional assumption that a fixed fraction of the means is known. We study the problem of variance estimation from a frequentist Bayesian perspective. The maximum likelihood estimator (MLE) for σ2 is biased and inconsistent. This raises the question whether the posterior is able to correct the MLE in this case. By developing a new proving strategy that uses refined properties of the posterior distribution, we find that the marginal posterior is inconsistent for any i.i.d. prior on the mean parameters. In particular, no assumption on the decay of the prior needs to be imposed. Surprisingly, we also find that consistency can be retained for a hierarchical prior based on Gaussian mixtures. In this case we also establish a limiting shape result and determine the limit distribution. In contrast to the classical Bernstein-von Mises theorem, the limit is non-Gaussian. We show that the Bayesian analysis leads to new statistical estimators outperforming the correctly calibrated MLE in a numerical simulation study.

AB - Consider the Gaussian sequence model under the additional assumption that a fixed fraction of the means is known. We study the problem of variance estimation from a frequentist Bayesian perspective. The maximum likelihood estimator (MLE) for σ2 is biased and inconsistent. This raises the question whether the posterior is able to correct the MLE in this case. By developing a new proving strategy that uses refined properties of the posterior distribution, we find that the marginal posterior is inconsistent for any i.i.d. prior on the mean parameters. In particular, no assumption on the decay of the prior needs to be imposed. Surprisingly, we also find that consistency can be retained for a hierarchical prior based on Gaussian mixtures. In this case we also establish a limiting shape result and determine the limit distribution. In contrast to the classical Bernstein-von Mises theorem, the limit is non-Gaussian. We show that the Bayesian analysis leads to new statistical estimators outperforming the correctly calibrated MLE in a numerical simulation study.

U2 - 10.1214/19-EJS1671

DO - 10.1214/19-EJS1671

M3 - Article

VL - 14

SP - 239

EP - 271

JO - Electronic Journal of Statistics

JF - Electronic Journal of Statistics

SN - 1935-7524

IS - 1

ER -