On the inability of Gaussian process regression to optimally learn compositional functions

Matteo Giordano; Kolyan Ray; Johannes Schmidt-Hieber

doi:10.5555/3600270

On the inability of Gaussian process regression to optimally learn compositional functions

Matteo Giordano, Kolyan Ray, Johannes Schmidt-Hieber

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

35 Downloads (Pure)

Abstract

We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.

Original language	English
Title of host publication	NIPS'22
Subtitle of host publication	Proceedings of the 36th International Conference on Neural Information Processing Systems
Editors	S. Koyejo
Place of Publication	Red Hook, NY
Publisher	Curran Associates Inc.
Pages	22341 -22353
ISBN (Print)	978-1-7138-7108-8
DOIs	https://doi.org/10.5555/3600270
Publication status	Published - 3 Apr 2024
Event	36th International Conference on Neural Information Processing Systems, NIPS 2022 - New Orleans, United States Duration: 2 Nov 2022 → 9 Dec 2022 Conference number: 36

Conference

Conference	36th International Conference on Neural Information Processing Systems, NIPS 2022
Abbreviated title	NIPS 2022
Country/Territory	United States
City	New Orleans
Period	2/11/22 → 9/12/22

Access to Document

10.5555/3600270

ArticleFinal published version, 301 KB

1 Conference contribution
1 Preprint

On the inability of Gaussian process regression to optimally learn compositional functions
Giordano, M., Ray, K. & Schmidt-Hieber, J., 16 May 2022, ArXiv.org, 20 p.
Research output: Working paper › Preprint › Academic

Open Access
File
34 Downloads (Pure)
On the inability of Gaussian process regression to optimally learn compositional functions
Giordano, M., Ray, K. & Schmidt-Hieber, J., 2022, 36th Conference on Neural Information Processing Systems, NeurIPS 2022. Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K. & Oh, A. (eds.). Neural information processing systems foundation, 13 p. (Advances in Neural Information Processing Systems; vol. 35).
Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Open Access
File

Cite this

@inproceedings{6e67833bb30e47c6940866f4736b15e2,

title = "On the inability of Gaussian process regression to optimally learn compositional functions",

abstract = "We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.",

author = "Matteo Giordano and Kolyan Ray and Johannes Schmidt-Hieber",

year = "2024",

month = apr,

day = "3",

doi = "10.5555/3600270",

language = "English",

isbn = "978-1-7138-7108-8",

pages = "22341 --22353",

editor = "S. Koyejo",

booktitle = "NIPS'22",

publisher = "Curran Associates Inc.",

address = "United States",

note = "36th International Conference on Neural Information Processing Systems, NIPS 2022, NIPS 2022 ; Conference date: 02-11-2022 Through 09-12-2022",

}

Giordano, M, Ray, K & Schmidt-Hieber, J 2024, On the inability of Gaussian process regression to optimally learn compositional functions. in S Koyejo (ed.), NIPS'22: Proceedings of the 36th International Conference on Neural Information Processing Systems., 1623, Curran Associates Inc., Red Hook, NY, pp. 22341 -22353, 36th International Conference on Neural Information Processing Systems, NIPS 2022, New Orleans, Louisiana, United States, 2/11/22. https://doi.org/10.5555/3600270

On the inability of Gaussian process regression to optimally learn compositional functions. / Giordano, Matteo; Ray, Kolyan; Schmidt-Hieber, Johannes.
NIPS'22: Proceedings of the 36th International Conference on Neural Information Processing Systems. ed. / S. Koyejo. Red Hook, NY: Curran Associates Inc., 2024. p. 22341 -22353 1623.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - On the inability of Gaussian process regression to optimally learn compositional functions

AU - Giordano, Matteo

AU - Ray, Kolyan

AU - Schmidt-Hieber, Johannes

N1 - Conference code: 36

PY - 2024/4/3

Y1 - 2024/4/3

N2 - We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.

AB - We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.

U2 - 10.5555/3600270

DO - 10.5555/3600270

M3 - Conference contribution

SN - 978-1-7138-7108-8

SP - 22341

EP - 22353

BT - NIPS'22

A2 - Koyejo, S.

PB - Curran Associates Inc.

CY - Red Hook, NY

T2 - 36th International Conference on Neural Information Processing Systems, NIPS 2022

Y2 - 2 November 2022 through 9 December 2022

ER -

On the inability of Gaussian process regression to optimally learn compositional functions

Abstract

Conference

Access to Document

Fingerprint

Research output