Activities per year
Abstract
We prove that the set of functions representable by ReLU neural networks with integer weights strictly increases with the network depth while allowing arbitrary width. More precisely, we show that dlog2(n)e hidden layers are indeed necessary to compute the maximum of n numbers, matching known upper bounds. Our results are based on the known duality between neural networks and Newton polytopes via tropical geometry. The integrality assumption implies that these Newton polytopes are lattice polytopes. Then, our depth lower bounds follow from a parity argument on the normalized volume of faces of such polytopes.
Original language | English |
---|---|
Number of pages | 13 |
Publication status | Published - 2023 |
Event | 11th International Conference on Learning Representations, ICLR 2023 - Kigali, Rwanda Duration: 1 May 2023 → 5 May 2023 Conference number: 11 |
Conference
Conference | 11th International Conference on Learning Representations, ICLR 2023 |
---|---|
Abbreviated title | ICLR 2023 |
Country/Territory | Rwanda |
City | Kigali |
Period | 1/05/23 → 5/05/23 |
Fingerprint
Dive into the research topics of 'Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes'. Together they form a unique fingerprint.Activities
- 1 Oral presentation
-
Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes
Haase, C. (Speaker), Hertrich, C. (Speaker) & Loho, G. (Speaker)
May 2023Activity: Talk or presentation › Oral presentation