Newest papers

Depth Separation for Neural Networks under Gaussian Measure

Cynthia Bortolotto, Francisco Ganacim, Roberto I. Oliveira · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28800 · cited by 0 works (OpenAlex) · open-access PDF

German

Wir überprüfen das Problem der Tiefentrennung für neuronale Netzwerke, wie von Eldan und Shamir, Daniely und anderen untersucht. Insbesondere betrachten wir die natürliche Einstellung der $L^2$ Annäherung von radialen Funktionen unter dem Standard Gaussian Maße in $d$ Dimensionen. Wir zeigen, dass es eine radiale Funktion in $L^2$ unter dem Standard Gaussian Maße gibt, die zu einem festen ausreichend kleinen $L^2$ Fehler durch ein Tiefe-3 ReLU neuronalen Netzwerk der Polynomengröße in $d$, aber kann nur durch Tiefe-2 Netzwerke mit subexponentiell begrenzten Gewichten annähernd sein, wenn ihre Größe mindestens exponentiell in $d$ ist.

Machine translation (Meta M2M100); the English original follows.

Abstract

We revisit the problem of depth separation for neural networks, as studied by Eldan and Shamir, Daniely, and others. Specifically, we consider the natural setting of $L^2$ approximation of radial functions under the standard Gaussian measure in $d$ dimensions. We show that there is a radial function in $L^2$ under the standard Gaussian measure that can be approximated to a fixed sufficiently small $L^2$ error by a depth-3 ReLU neural network of polynomial size in $d$ but can only be approximated by depth-2 networks with subexponentially bounded weights if their size is at least exponential in $d$. For that, we prove a general result showing that if a radial function has substantial mass in the high frequencies of its Hermite expansion, then it cannot be well approximated by a depth-2 neural network with small width and bounded weights. All theorems, lemmas, propositions, and corollaries are verified in Lean~4.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Bounded function, Measure (data warehouse), Mathematics, Artificial neural network, Gaussian, Hermite polynomials

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange