Newest papers

Convergence of stochastic gradient descent for learning linear neural networks

Gabin Maxime Nguegnang, Bubacarr Bah, Holger Rauhut, Ulrich Terstiege · 2026 · arXiv

DOI: 10.48550/arXiv.2610.28561 · open-access PDF

Indonesian

Stochastic gradient descent (SGD) adalah batu penjuru pembelajaran mendalam modern, namun sifat-sifat konvergensi teoritisnya masih kurang dipahami dalam pengaturan non-convex. Untuk mengatasi kesenjangan ini, kami mempelajari dinamika SGD dalam jaringan saraf linier mendalam - model disederhanakan yang mempertahankan karakteristik non-convex kunci sambil tetap dapat dianalisis secara analitis. Berdasarkan hasil baru-baru ini untuk gradient descent, kami memperluas jaminan konvergensi ke SGD di bawah kerugian persegi yang diatur. Kami menetapkan batas hampir pasti dari iterasi SGD. Menggunakan kerangka kerja pseudotrajectories asymptotic dari stochastic approxima

Machine translation (Meta M2M100); the English original follows.

Abstract

Stochastic gradient descent (SGD) is the cornerstone of modern deep learning, yet its theoretical convergence properties remain poorly understood in non-convex settings. To address this gap, we study SGD dynamics in deep linear neural networks - simplified models that retain key non-convex characteristics while remaining analytically tractable. Building on recent results for gradient descent, we extend convergence guarantees to SGD under regularized squared loss. We establish almost sure boundedness of SGD iterates. Using the framework of asymptotic pseudotrajectories from stochastic approxima

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange