On the inability of Gaussian process regression to optimally learn compositional functions

Matteo Giordano; Kolyan Ray; Johannes Schmidt-Hieber

On the inability of Gaussian process regression to optimally learn compositional functions

Matteo Giordano, Kolyan Ray, Johannes Schmidt-Hieber

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

3 Citations (Scopus)

105 Downloads (Pure)

Abstract

We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.

Original language	English
Title of host publication	36th Conference on Neural Information Processing Systems, NeurIPS 2022
Editors	S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh
Publisher	Neural information processing systems foundation
Number of pages	13
ISBN (Electronic)	9781713871088
Publication status	Published - 2022
Event	36th Annual Conference on Neural Information Processing Systems, NeurIPS 2022: Connecting Methods and Applications - New Orleans Convention Center, New Orleans, United States Duration: 28 Nov 2022 → 9 Dec 2022 Conference number: 36 https://neurips.cc/Conferences/2022

Publication series

Name	Advances in Neural Information Processing Systems
Publisher	Neural Information Processing Systems Foundation
Volume	35
ISSN (Print)	1049-5258

Conference

Conference	36th Annual Conference on Neural Information Processing Systems, NeurIPS 2022
Abbreviated title	NeurIPS 2022
Country/Territory	United States
City	New Orleans
Period	28/11/22 → 9/12/22
Internet address	https://neurips.cc/Conferences/2022

Access to Document

ArticleFinal published version, 301 KB

https://papers.nips.cc/paper_files/paper/2022/hash/8c420176b45e923cf99dee1d7356a763-Abstract-Conference.html

Cite this

Giordano, M., Ray, K., & Schmidt-Hieber, J. (2022). On the inability of Gaussian process regression to optimally learn compositional functions. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, & A. Oh (Eds.), 36th Conference on Neural Information Processing Systems, NeurIPS 2022 (Advances in Neural Information Processing Systems; Vol. 35). Neural information processing systems foundation. https://papers.nips.cc/paper_files/paper/2022/hash/8c420176b45e923cf99dee1d7356a763-Abstract-Conference.html

Giordano, Matteo ; Ray, Kolyan ; Schmidt-Hieber, Johannes. / On the inability of Gaussian process regression to optimally learn compositional functions. 36th Conference on Neural Information Processing Systems, NeurIPS 2022. editor / S. Koyejo ; S. Mohamed ; A. Agarwal ; D. Belgrave ; K. Cho ; A. Oh. Neural information processing systems foundation, 2022. (Advances in Neural Information Processing Systems).

@inproceedings{2ec6472240e04df5a2c144a325b96d4c,

title = "On the inability of Gaussian process regression to optimally learn compositional functions",

abstract = "We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.",

author = "Matteo Giordano and Kolyan Ray and Johannes Schmidt-Hieber",

note = "Publisher Copyright: {\textcopyright} 2022 Neural information processing systems foundation. All rights reserved.; 36th Annual Conference on Neural Information Processing Systems, NeurIPS 2022 : Connecting Methods and Applications, NeurIPS 2022 ; Conference date: 28-11-2022 Through 09-12-2022",

year = "2022",

language = "English",

series = "Advances in Neural Information Processing Systems",

publisher = "Neural information processing systems foundation",

editor = "S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh",

booktitle = "36th Conference on Neural Information Processing Systems, NeurIPS 2022",

address = "United States",

url = "https://neurips.cc/Conferences/2022",

}

Giordano, M, Ray, K & Schmidt-Hieber, J 2022, On the inability of Gaussian process regression to optimally learn compositional functions. in S Koyejo, S Mohamed, A Agarwal, D Belgrave, K Cho & A Oh (eds), 36th Conference on Neural Information Processing Systems, NeurIPS 2022. Advances in Neural Information Processing Systems, vol. 35, Neural information processing systems foundation, 36th Annual Conference on Neural Information Processing Systems, NeurIPS 2022, New Orleans, Louisiana, United States, 28/11/22. <https://papers.nips.cc/paper_files/paper/2022/hash/8c420176b45e923cf99dee1d7356a763-Abstract-Conference.html>

On the inability of Gaussian process regression to optimally learn compositional functions. / Giordano, Matteo; Ray, Kolyan; Schmidt-Hieber, Johannes.
36th Conference on Neural Information Processing Systems, NeurIPS 2022. ed. / S. Koyejo; S. Mohamed; A. Agarwal; D. Belgrave; K. Cho; A. Oh. Neural information processing systems foundation, 2022. (Advances in Neural Information Processing Systems; Vol. 35).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - On the inability of Gaussian process regression to optimally learn compositional functions

AU - Giordano, Matteo

AU - Ray, Kolyan

AU - Schmidt-Hieber, Johannes

N1 - Conference code: 36

PY - 2022

Y1 - 2022

N2 - We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.

AB - We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.

UR - http://www.scopus.com/inward/record.url?scp=85163174423&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85163174423

T3 - Advances in Neural Information Processing Systems

BT - 36th Conference on Neural Information Processing Systems, NeurIPS 2022

A2 - Koyejo, S.

A2 - Mohamed, S.

A2 - Agarwal, A.

A2 - Belgrave, D.

A2 - Cho, K.

A2 - Oh, A.

PB - Neural information processing systems foundation

T2 - 36th Annual Conference on Neural Information Processing Systems, NeurIPS 2022

Y2 - 28 November 2022 through 9 December 2022

ER -

Giordano M, Ray K, Schmidt-Hieber J. On the inability of Gaussian process regression to optimally learn compositional functions. In Koyejo S, Mohamed S, Agarwal A, Belgrave D, Cho K, Oh A, editors, 36th Conference on Neural Information Processing Systems, NeurIPS 2022. Neural information processing systems foundation. 2022. (Advances in Neural Information Processing Systems).

On the inability of Gaussian process regression to optimally learn compositional functions

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint