Newest papers

Weak-$L^α$ Noise, Strong Guarantees: Gradient Clipping under Affine Heavy Tails

Alyona Nekrasova, Makar Sidorov, Savelii Chezhegov, Alexander Gasnikov, Margarita Preobrazhenskaia, Igor Preobrazhensky, Eduard Gorbunov · 2026 · arXiv

DOI: 10.48550/arXiv.2610.28107 · open-access PDF

Chinese

Gradient clipping 是获取高概率汇总保证的关键工具,用于在重尾噪音下使用斯托哈斯特式第一类方法。现有分析通常假定订单的中心时刻为 $α\in(1,2)$,这不包括在关键指数上订单的尾巴为 $u^{-α}$的分布。在这篇论文中,我们考虑在敏感的重尾噪音下进行平滑的曲线优化,其规模可能与降级标准一起增长。我们从削减的 SGD (Clip-SGD) 和其加速变体 Clip-SSTM 在弱$L^$尾巴条件下获得高概率汇总保证,即

Machine translation (Meta M2M100); the English original follows.

Abstract

Gradient clipping is a key tool for obtaining high-probability convergence guarantees for stochastic first-order methods under heavy-tailed noise. Existing analyses typically assume a bounded central moment of order $α\in(1,2]$, which excludes distributions with tails of order $u^{-α}$ at the critical exponent. In this paper, we consider smooth convex optimization under affine heavy-tailed noise, whose scale may grow with the gradient norm. We derive high-probability convergence guarantees for clipped SGD (Clip-SGD) and its accelerated variant Clip-SSTM under a weak-$L^α$ tail condition, which

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange