Newest papers

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

Youran Dong, Junfeng Yang, Wei Yao, Jin Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26839 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

La optimización de Bilevel proporciona un marco de datos para tareas de aprendizaje de máquinas como la optimización de hiperparámetros, la ponderación de datos y el meta-aprendizaje. Los métodos de bilevel existentes basados en Newton suelen depender de la fórmula hipergradiente clásica, que generalmente no está disponible cuando las soluciones de nivel inferior son no únicas y el nivel inferior de Hessian es singular. Desarrollamos dos métodos basados en Newton más allá de la convexidad fuerte de nivel inferior aprovechando la curvatura compartida por las actualizaciones de síntesis de nivel inferior y de multiplicador. El GALET basado en Newton maneja la curvatura de nivel inferior potencialmente singular bajo condiciones espectrales positivas, aprovechando su estructura de rango para evitar el sistema de multiplicador

Machine translation (Meta M2M100); the English original follows.

Abstract

Bilevel optimization provides a framework for machine learning tasks such as hyperparameter optimization, data reweighting, and meta-learning. Existing Newton-based bilevel methods typically rely on the classical hypergradient formula, which is generally unavailable when lower-level solutions are nonunique and the lower-level Hessian is singular. We develop two Newton-based methods beyond lower-level strong convexity by exploiting curvature shared by the lower-level and multiplier updates. Newton-based GALET handles potentially singular lower-level curvature under a positive spectral-gap condition, exploiting its range structure to avoid the squared-Hessian multiplier system used in GALET. Newton-based BAMM uses a strongly convex aggregation model when the lower-level objective is convex and the upper-level objective is strongly convex in the lower-level variable. We establish convergence guarantees and improved complexity bounds relative to GALET and sl-BAMM under the corresponding assumptions. We further introduce GNBO, a generalized Newton-based framework that captures the shared-curvature structure of the proposed methods and several existing Newton and quasi-Newton bilevel methods. Experiments on synthetic problems, data hyper-cleaning, and GPT-2 data reweighting evaluate the proposed methods across different bilevel learning tasks.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Convexity, Bilevel optimization, Hessian matrix, Mathematical optimization, Mathematics, Hyperparameter

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange