Newest papers

Depth Separation for Neural Networks under Gaussian Measure

Cynthia Bortolotto, Francisco Ganacim, Roberto I. Oliveira · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28800 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Revisamos el problema de la separación de profundidad para las redes neuronales, como lo estudiaron Eldan y Shamir, Daniely, y otros. En concreto, consideramos el ajuste natural de la aproximación de $L^2$ de las funciones radiales bajo la medida estándar de Gauss en dimensiones de $d$. Mostramos que hay una función radial en $L^2$ bajo la medida estándar de Gauss que puede ser aproximada a un error suficientemente pequeño de $L^2$ por una red neural de profundidad-3 ReLU de tamaño polinómico en $d$, pero sólo puede ser aproximada por redes de profundidad-2 con pesos subexponencialmente limitados si su tamaño es al menos exponencial en $d$. Para ello, demostramos un resultado general que si una función radial tiene masa sustancial en las altas

Machine translation (Meta M2M100); the English original follows.

Abstract

We revisit the problem of depth separation for neural networks, as studied by Eldan and Shamir, Daniely, and others. Specifically, we consider the natural setting of $L^2$ approximation of radial functions under the standard Gaussian measure in $d$ dimensions. We show that there is a radial function in $L^2$ under the standard Gaussian measure that can be approximated to a fixed sufficiently small $L^2$ error by a depth-3 ReLU neural network of polynomial size in $d$ but can only be approximated by depth-2 networks with subexponentially bounded weights if their size is at least exponential in $d$. For that, we prove a general result showing that if a radial function has substantial mass in the high frequencies of its Hermite expansion, then it cannot be well approximated by a depth-2 neural network with small width and bounded weights. All theorems, lemmas, propositions, and corollaries are verified in Lean~4.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Bounded function, Measure (data warehouse), Mathematics, Artificial neural network, Gaussian, Hermite polynomials

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange