Newest papers

Variance Reduction with Delayed Feedback for Non-Convex Optimization

Wei Jiang, Dingzhi Yu, Zechao Li · 2026 · arXiv

DOI: 10.48550/arXiv.2610.28510 · open-access PDF

Chinese

我们研究非缩优化的差异减少,当差异评估延迟返回时。服务器可以更新参数,而以前的评估仍然未完成。我们首先分析一个顺序的回归动力评估器,用于斯托加斯蒂克优化。在平均平滑和边界差异下,它的口号复杂度为 $O(1+σ^2ε^-2}+LΔ_f(1+σ/ε+τ)ε^-2}$,目标是最多 $ε^2$和 $τ$限制了反馈延迟的预期平方差距标准。我们接下来研究 $n$组件的有限总和获得 $O(n+L_Δf(\sqrt n+min\{{,n-1}\,ε^-2})$评估。

Machine translation (Meta M2M100); the English original follows.

Abstract

We study variance reduction for non-convex optimization when gradient evaluations return with a delay. The server can update the parameter while earlier evaluations remain unfinished. We first analyze an ordered recursive momentum estimator for stochastic optimization. Under average smoothness and bounded variance, its oracle complexity is $O(1+σ^2ε^{-2}+LΔ_f(1+σ/ε+τ)ε^{-2})$, where the target is an expected squared gradient norm of at most $ε^2$ and $τ$ bounds the feedback delay. We next study finite sums with $n$ components and obtain $O(n+LΔ_f(\sqrt n+\min\{τ,n-1\})ε^{-2})$ evaluations. An

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange