Newest papers

Variance Reduction with Delayed Feedback for Non-Convex Optimization

Wei Jiang, Dingzhi Yu, Zechao Li · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28510 · cited by 0 works (OpenAlex) · open-access PDF

Korean

서버는 이전 평가가 완료되지 않은 동안 매개 변수를 업데이트할 수 있습니다. 우리는 먼저 스토카스틱 최적화를위한 순서대로 복구 변수를 분석합니다. 평균 부드러움과 한계 변수 아래, 그 오라클 복잡성은 $O(1+σ^2ε^-2}+LΔ_f(1+σ/ε+τ)ε^-2})$이며, 목표는 최대 $ε^2$ 및 $τ$의 예상 사각 변수 규범입니다. 우리는 다음으로 $n$ 구성 요소와 최종 금액을 연구하고 $O(n+L_Δf(\sqrt n\\min\n{n-1}\n-2})$ 평가를 얻습니다. Anynchronous snapshot 변형은 명확한 경계 아래의 전체 컴퓨팅을 통해 계속 업데이트

Machine translation (Meta M2M100); the English original follows.

Abstract

We study variance reduction for non-convex optimization when gradient evaluations return with a delay. The server can update the parameter while earlier evaluations remain unfinished. We first analyze an ordered recursive momentum estimator for stochastic optimization. Under average smoothness and bounded variance, its oracle complexity is $O(1+σ^2ε^{-2}+LΔ_f(1+σ/ε+τ)ε^{-2})$, where the target is an expected squared gradient norm of at most $ε^2$ and $τ$ bounds the feedback delay. We next study finite sums with $n$ components and obtain $O(n+LΔ_f(\sqrt n+\min\{τ,n-1\})ε^{-2})$ evaluations. An asynchronous snapshot variant continues updating during full-gradient computations under an explicit bound on their completion time. For the stochastic method, we further replace a maximum-delay bound by a bound on the weighted amount of missing feedback.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Mathematics, Estimator, Upper and lower bounds, Bounded function, Variance reduction, Smoothness

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange