Newest papers

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

Youran Dong, Junfeng Yang, Wei Yao, Jin Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26839 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

Bilevelの最適化は、ハイパーパラメータの最適化、データ重量化、メタ学習などの機械学習タスクのためのデータフレームワークを提供します。既存のNewtonベースのBalevel方法は、通常、低レベルのソリューションが非ユニークで、低レベルのHessianがユニークである場合のクラシックのハイパーグラディン式に依存し、一般的に利用できません。私たちは、低レベルの強力なコンベクシティーを超える2つのNewtonベースの方法を開発します。NewtonベースのGALETは、低レベルのオブジェクトがコンベクシアムであり、上レベルのオブジェクトは低レベルの変数で強力なコンベクシアムを

Machine translation (Meta M2M100); the English original follows.

Abstract

Bilevel optimization provides a framework for machine learning tasks such as hyperparameter optimization, data reweighting, and meta-learning. Existing Newton-based bilevel methods typically rely on the classical hypergradient formula, which is generally unavailable when lower-level solutions are nonunique and the lower-level Hessian is singular. We develop two Newton-based methods beyond lower-level strong convexity by exploiting curvature shared by the lower-level and multiplier updates. Newton-based GALET handles potentially singular lower-level curvature under a positive spectral-gap condition, exploiting its range structure to avoid the squared-Hessian multiplier system used in GALET. Newton-based BAMM uses a strongly convex aggregation model when the lower-level objective is convex and the upper-level objective is strongly convex in the lower-level variable. We establish convergence guarantees and improved complexity bounds relative to GALET and sl-BAMM under the corresponding assumptions. We further introduce GNBO, a generalized Newton-based framework that captures the shared-curvature structure of the proposed methods and several existing Newton and quasi-Newton bilevel methods. Experiments on synthetic problems, data hyper-cleaning, and GPT-2 data reweighting evaluate the proposed methods across different bilevel learning tasks.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Convexity, Bilevel optimization, Hessian matrix, Mathematical optimization, Mathematics, Hyperparameter

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange