Function Approximation by Deep Neural Networks with Parameters $$\{0,\pm \frac{1}{2}, \pm 1, 2\}$$

In this paper, it is shown that Cβ-smooth functions can be approximated by deep neural networks with ReLU activation function and with parameters {0,±12,±1,2}
. The l0 and l1 parameter norms of considered networks are thus equivalent. The depth, the width and the number of active parameters of the constructed networks have, up to a logarithmic factor, the same dependence on the approximation error as the networks with parameters in [−1,1]. In particular, this implies that the nonparametric regression estimation with constructed networks achieves, up to logarithmic factors, the same minimax convergence rates as with sparse networks with parameters in [−1,1].