Activities per year
Abstract
The approximation properties of infinitely wide shallow neural networks heavily depend on the choice of the activation function. To understand this influence, we study embeddings between Barron spaces with different activation functions. These embeddings are proven by providing pushforward maps on the measures $\mu$ used to represent functions $f$. An activation function of particular interest is the rectified power unit ($\operatorname{RePU}$) given by $\operatorname{RePU}_s(x)=\max(0,x)^s$. For many commonly used activation functions, the wellknown Taylor remainder theorem can be used to construct a pushforward map, which allows us to prove the embedding of the associated Barron space into a Barron space with a $\operatorname{RePU}$ as activation function. Moreover, the Barron spaces associated with the $\operatorname{RePU}_s$ have a hierarchical structure similar to the Sobolev spaces $H^m$.
Original language  English 

DOIs  
Publication status  Published  25 May 2023 
Keywords
 stat.ML
 cs.LG
 math.FA
 46E35 (Primary) 46E15, 46G12 (Secondary)
 I.2.6; G.1.9
Fingerprint
Dive into the research topics of 'Embeddings between Barron spaces with higher order activation functions'. Together they form a unique fingerprint.Activities
 2 Oral presentation

Learning a Sparse Representation of Barron Functions with the Inverse Scale Space Flow
Heeringa, T. (Speaker)
15 May 2024Activity: Talk or presentation › Oral presentation

Embeddings between Barron spaces with higher order activation functions
Heeringa, T. (Speaker)
1 Mar 2023Activity: Talk or presentation › Oral presentation