Newest papers

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

Youran Dong, Junfeng Yang, Wei Yao, Jin Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26839 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

A otimização de Bilevel fornece um quadro de dados para tarefas de aprendizagem de dados como otimização de hiperparâmetros, ponderação de dados e meta-aprendizagem. Os métodos de bilevel existentes baseados em Newton geralmente dependem da fórmula hipergradiente clássica, que geralmente não está disponível quando as soluções de nível inferior são não-únicas e o Hessian de nível inferior é único. Desenvolvemos dois métodos baseados em Newton para além da convexidade forte de nível inferior, explorando a curvatura compartilhada pelas atualizações de síntese de nível inferior e de multiplicador biliar. O GALET baseado em Newton lida com curvatura de nível inferior potencialmente singular sob uma condição de diferença espectral positiva, explorando sua estrutura de intervalo para evitar o sistema de multi

Machine translation (Meta M2M100); the English original follows.

Abstract

Bilevel optimization provides a framework for machine learning tasks such as hyperparameter optimization, data reweighting, and meta-learning. Existing Newton-based bilevel methods typically rely on the classical hypergradient formula, which is generally unavailable when lower-level solutions are nonunique and the lower-level Hessian is singular. We develop two Newton-based methods beyond lower-level strong convexity by exploiting curvature shared by the lower-level and multiplier updates. Newton-based GALET handles potentially singular lower-level curvature under a positive spectral-gap condition, exploiting its range structure to avoid the squared-Hessian multiplier system used in GALET. Newton-based BAMM uses a strongly convex aggregation model when the lower-level objective is convex and the upper-level objective is strongly convex in the lower-level variable. We establish convergence guarantees and improved complexity bounds relative to GALET and sl-BAMM under the corresponding assumptions. We further introduce GNBO, a generalized Newton-based framework that captures the shared-curvature structure of the proposed methods and several existing Newton and quasi-Newton bilevel methods. Experiments on synthetic problems, data hyper-cleaning, and GPT-2 data reweighting evaluate the proposed methods across different bilevel learning tasks.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Convexity, Bilevel optimization, Hessian matrix, Mathematical optimization, Mathematics, Hyperparameter

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange