Newest papers

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

Youran Dong, Junfeng Yang, Wei Yao, Jin Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26839 · cited by 0 works (OpenAlex) · open-access PDF

Korean

Bilevel 최적화는 하이퍼파라미터 최적화, 데이터 재중량 및 메타 학습과 같은 기계 학습 작업에 대한 데이터 프레임 워크를 제공합니다. 기존의 Newton 기반의 bilevel 방법은 일반적으로 낮은 수준의 솔루션이 비유적이고 낮은 수준의 Hessian이 단일일 때 일반적으로 사용할 수없는 고전적인 hypergradient 수식에 의존합니다. 우리는 낮은 수준의 강한 유연성 이외의 두 개의 Newton 기반 방법을 개발합니다. Newton 기반의 GALET은 긍정적 인 스펙트럼 격차 조건에서 잠재적으로 단일 낮은 수준의 유연성을 처리하며 범위 구조를 활용하여 GALET에서 사용되는 사각형-Hessian 다수 시스템을 피합니다. Newton 기반의 BAMM는 낮은 수준의

Machine translation (Meta M2M100); the English original follows.

Abstract

Bilevel optimization provides a framework for machine learning tasks such as hyperparameter optimization, data reweighting, and meta-learning. Existing Newton-based bilevel methods typically rely on the classical hypergradient formula, which is generally unavailable when lower-level solutions are nonunique and the lower-level Hessian is singular. We develop two Newton-based methods beyond lower-level strong convexity by exploiting curvature shared by the lower-level and multiplier updates. Newton-based GALET handles potentially singular lower-level curvature under a positive spectral-gap condition, exploiting its range structure to avoid the squared-Hessian multiplier system used in GALET. Newton-based BAMM uses a strongly convex aggregation model when the lower-level objective is convex and the upper-level objective is strongly convex in the lower-level variable. We establish convergence guarantees and improved complexity bounds relative to GALET and sl-BAMM under the corresponding assumptions. We further introduce GNBO, a generalized Newton-based framework that captures the shared-curvature structure of the proposed methods and several existing Newton and quasi-Newton bilevel methods. Experiments on synthetic problems, data hyper-cleaning, and GPT-2 data reweighting evaluate the proposed methods across different bilevel learning tasks.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Convexity, Bilevel optimization, Hessian matrix, Mathematical optimization, Mathematics, Hyperparameter

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange