Newest papers

Fine-Grained Analysis of Catastrophic Forgetting in Continual Learning via Algorithmic Stability

Guodong Zheng, Yan Sun, Peng Wang, Enneng Yang, Jiafei Lyu, Xiaochun Cao, Dacheng Tao, Li Shen · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31551 · cited by 0 works (OpenAlex) · open-access PDF

French

Comprendre l’oubli catastrophique de FinNet, dans lequel l’apprentissage de nouvelles tâches dégrade les performances sur les tâches déjà apprises, reste un défi fondamental dans l’apprentissage continu (CL). Les analyses théoriques existantes reposent souvent sur des modèles ou des structures de tâches spécifiques et offrent un aperçu limité de la façon dont l’optimisation itérative affecte l’oubli. Pour combler ce fossé, nous développons un cadre algorithmique de stabilité pour l’oubli de la population prévue sous SGD sans prescrire une architecture de modèle spécifique. Spécifiquement, nous décomposons l’oubli dans des horaires d’apprentissage intra-task et inter-task et des horaires

Machine translation (Meta M2M100); the English original follows.

Abstract

Understanding catastrophic forgetting, in which learning new tasks degrades performance on previously learned tasks, remains a fundamental challenge in continual learning (CL). Existing theoretical analyses often rely on specific model or task structures and offer limited insight into how iterative optimization affects forgetting. To address this gap, we develop an algorithmic stability framework for expected population forgetting under SGD without prescribing a particular model architecture. Specifically, we decompose forgetting into intra-task and cross-task generalization gaps and empirical forgetting. To bound the generalization gaps, we track how a sample perturbation affects updates within a task and propagates through subsequent tasks. We consider constant, intra-task decaying, and continuously decaying learning-rate schedules. Comparing the two decaying schedules allows us to characterize how restarting learning-rate decay at task boundaries affects the propagation of historical sample perturbations. For empirical forgetting, we derive bounds for all three schedules by retaining cumulative gradient alignment along the optimization trajectory, without requiring convexity. Combining these results yields upper bounds on expected forgetting that capture the effects of training set size, the number of updates per task, the total number of tasks, learning-rate schedules, and cumulative gradient alignment. Finally, we conduct experiments on Split CIFAR-100 and Split MiniImageNet to examine the empirical implications of our theoretical findings.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Forgetting, Generalization, Computer science, Stability (learning theory), Task (project management), Set (abstract data type)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange