Newest papers

Adaptive Normalized Gradient Method for $(L_0,L_1)$-smooth Problems: Estimating Problem Parameters as a Region

R. Okazaki, Shotaro Yagishita, Hideitsu Hino · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.27681 · cited by 0 works (OpenAlex) · open-access PDF

Korean

우리는 $(L_0,L_1)$-smooth 최적화에 대한 파라미터-free 및 효율적인 gradient descent 알고리즘을 제안합니다. 이 유연성 개념은 Zhang et al. (2020)에 의해 도입되었으며, gradient 규범과 함께 성장하는 유연성을 허용합니다. 최첨단 위치와 마찬가지로, 우리의 방법은 $(L_0,L_1)$에 대한 확실한 값의 영역을 유지하기 위해 관찰의 시퀀스를 사용하고 단일 단계 크기를 선택하기 위해이 영역을 사용합니다. 유연성 설정에서, 우리의 방법은 $\de\mathcal_O_left에 $\left(L_0/ε+ L_1^2\right)의 왼쪽에있는 표준 이더링 복잡성을 달성합니다

Machine translation (Meta M2M100); the English original follows.

Abstract

We propose a parameter-free and efficient gradient descent algorithm for $(L_0,L_1)$-smooth optimization. This notion of smoothness, introduced by Zhang et al. (2020), allows the smoothness bound to grow with the gradient norm. Similar to cutting-plane localization, our method uses a sequence of observations to maintain a region of plausible values for $(L_0,L_1)$ and uses this region to select step sizes. In the convex setting, our method attains the standard iteration complexity bound $\tilde{\mathcal{O}}\left(\left(L_0/ε+ L_1^2\right)\|x^0-x^*\|^2\right)$ for finding an $ε$-optimal solution, while in the nonconvex setting it finds an $ε$-stationary point within $\tilde{\mathcal{O}}\left(\left(L_0/ε^2 + L_1/ε\right)(f(x^0)-f^*)\right)$ iterations. To the best of our knowledge, this is the first work to establish, for a single parameter-free algorithm, the best-known iteration-complexity guarantee in the nonconvex setting while also providing the above iteration-complexity guarantee in the convex setting. Experiments on logistic regression and real phase retrieval demonstrate that our method can exploit $(L_0,L_1)$-smoothness and perform efficiently on nonconvex problems.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Smoothness, Gradient descent, Mathematics, Regular polygon, Mathematical optimization, Gradient method

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange