Newest papers

Depth Separation for Neural Networks under Gaussian Measure

Cynthia Bortolotto, Francisco Ganacim, Roberto I. Oliveira · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28800 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Kami meninjau kembali masalah pemisahan kedalaman untuk jaringan saraf, seperti yang dipelajari oleh Eldan dan Shamir, Daniely, dan lain-lain. Secara khusus, kami mempertimbangkan pengaturan alami dari $L^2$ mendekati fungsi radial di bawah ukuran standar Gaussian dalam dimensi $d$. Kami menunjukkan bahwa ada fungsi radial di $L^2$ di bawah ukuran standar Gaussian yang dapat didekati dengan kesalahan cukup kecil $L^2$ yang tetap oleh jaringan saraf ReLU kedalaman-3 dengan ukuran polinomial di $d$ tetapi hanya dapat didekati oleh jaringan kedalaman-2 dengan beban subeksponensial jika ukuran mereka setidaknya eksponensial di $d$. Untuk itu, kami membuktikan sebuah demonstrasi umum yang menunjukkan bahwa jika suatu fungsi radial memiliki massa yang substansial dalam frekuensi tinggi ekspansi Hermite, maka

Machine translation (Meta M2M100); the English original follows.

Abstract

We revisit the problem of depth separation for neural networks, as studied by Eldan and Shamir, Daniely, and others. Specifically, we consider the natural setting of $L^2$ approximation of radial functions under the standard Gaussian measure in $d$ dimensions. We show that there is a radial function in $L^2$ under the standard Gaussian measure that can be approximated to a fixed sufficiently small $L^2$ error by a depth-3 ReLU neural network of polynomial size in $d$ but can only be approximated by depth-2 networks with subexponentially bounded weights if their size is at least exponential in $d$. For that, we prove a general result showing that if a radial function has substantial mass in the high frequencies of its Hermite expansion, then it cannot be well approximated by a depth-2 neural network with small width and bounded weights. All theorems, lemmas, propositions, and corollaries are verified in Lean~4.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Bounded function, Measure (data warehouse), Mathematics, Artificial neural network, Gaussian, Hermite polynomials

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange