Newest papers

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

Youran Dong, Junfeng Yang, Wei Yao, Jin Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26839 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Bilevel optimization menyediakan kerangka kerja data untuk tugas pembelajaran mesin seperti optimasi hyperparameter, penilaian data, dan meta-learning. Metode bilevel berbasis Newton yang ada biasanya bergantung pada rumus hypergradient klasik, yang umumnya tidak tersedia ketika solusi tingkat bawah tidak unik dan Hessian tingkat bawah adalah unik. Kami mengembangkan dua metode berbasis Newton di luar konveksi kuat tingkat bawah dengan memanfaatkan kurva yang dibagikan oleh pembaruan tingkat bawah dan multiplier. GALET berbasis Newton menangani kurva tingkat bawah potensial yang unik di bawah kondisi spektral yang positif, mengeksploitasi struktur rentangnya untuk menghindari sistem multiplier Hessian persegi digunakan di GALET. BAMM berbasis Newton menggunakan model evaluasi tugas konveksi yang kuat ketika proposal tingkat bawah yang ada konveksi dan objektif tingkat atas sangat konveksi

Machine translation (Meta M2M100); the English original follows.

Abstract

Bilevel optimization provides a framework for machine learning tasks such as hyperparameter optimization, data reweighting, and meta-learning. Existing Newton-based bilevel methods typically rely on the classical hypergradient formula, which is generally unavailable when lower-level solutions are nonunique and the lower-level Hessian is singular. We develop two Newton-based methods beyond lower-level strong convexity by exploiting curvature shared by the lower-level and multiplier updates. Newton-based GALET handles potentially singular lower-level curvature under a positive spectral-gap condition, exploiting its range structure to avoid the squared-Hessian multiplier system used in GALET. Newton-based BAMM uses a strongly convex aggregation model when the lower-level objective is convex and the upper-level objective is strongly convex in the lower-level variable. We establish convergence guarantees and improved complexity bounds relative to GALET and sl-BAMM under the corresponding assumptions. We further introduce GNBO, a generalized Newton-based framework that captures the shared-curvature structure of the proposed methods and several existing Newton and quasi-Newton bilevel methods. Experiments on synthetic problems, data hyper-cleaning, and GPT-2 data reweighting evaluate the proposed methods across different bilevel learning tasks.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Convexity, Bilevel optimization, Hessian matrix, Mathematical optimization, Mathematics, Hyperparameter

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange