TY - JOUR
T1 - A supervised deep learning method for nonparametric density estimation
AU - Bos, Thijs
AU - Schmidt-Hieber, Johannes
N1 - Publisher Copyright:
© 2024, Institute of Mathematical Statistics. All rights reserved.
Financial transaction number:
6100042918
PY - 2024
Y1 - 2024
N2 - Nonparametric density estimation is an unsupervised learning problem. In this work we propose a two-step procedure that casts the density estimation problem in the first step into a supervised regression prob-lem. The advantage is that we can afterwards apply supervised learning methods. Compared to the standard nonparametric regression setting, the proposed procedure creates, however, dependence among the training sam-ples. To derive statistical risk bounds, one can therefore not rely on the well-developed theory for i.i.d. data. To overcome this, we prove an oracle inequality for this specific form of data dependence. As an application, it is shown that under a compositional structure assumption on the underlying density, the proposed two-step method achieves convergence rates that are faster than the standard nonparametric rates. A simulation study illus-trates the finite sample performance.
AB - Nonparametric density estimation is an unsupervised learning problem. In this work we propose a two-step procedure that casts the density estimation problem in the first step into a supervised regression prob-lem. The advantage is that we can afterwards apply supervised learning methods. Compared to the standard nonparametric regression setting, the proposed procedure creates, however, dependence among the training sam-ples. To derive statistical risk bounds, one can therefore not rely on the well-developed theory for i.i.d. data. To overcome this, we prove an oracle inequality for this specific form of data dependence. As an application, it is shown that under a compositional structure assumption on the underlying density, the proposed two-step method achieves convergence rates that are faster than the standard nonparametric rates. A simulation study illus-trates the finite sample performance.
KW - (Un)supervised learning
KW - Neural networks
KW - Nonparametric density esti-mation
KW - Statistical estimation rates
UR - http://www.scopus.com/inward/record.url?scp=85213570160&partnerID=8YFLogxK
U2 - 10.1214/24-EJS2332
DO - 10.1214/24-EJS2332
M3 - Article
AN - SCOPUS:85213570160
SN - 1935-7524
VL - 18
SP - 5601
EP - 5658
JO - Electronic Journal of Statistics
JF - Electronic Journal of Statistics
IS - 2
ER -