Newest papers

HERD: Harness-Evolving Recursive Distillation for Continual Learning

Xun Sun, Thanh Thi Nguyen, Thuy-Trang Vu · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31401 · cited by 0 works (OpenAlex) · open-access PDF

Korean

지속적인 학습은 모델이 기존 능력을 위협하지 않고도 새로운 기술과 지식을 지속적으로 습득하도록 요구합니다. 파라메트릭 방법은 재앙적 잊어버림과 플라스틱의 손실으로 고통받습니다. 프롬프트, 기술 및 헬렌스와 같은 비 파라메트릭 방법은 컨텍스트 길이 및 손실 된 평범한 텍스트 압축에 의해 제한되지 않지만 컨텍스트 학습을 효율적으로 활용합니다. 패턴을 효과적으로 결합하기 위해 HERD (Harness-Evolving Recursive Distillation)는 효율적으로 헬렌스에서 학습하고 누적 능력으로 경험을 내부화합니다. 헬렌스 측면에서 HERD는 재사용 가능한 패턴을 캡처하기위한 기술을 지속적으로 향상시키고 기술이

Machine translation (Meta M2M100); the English original follows.

Abstract

Continual learning requires a model to keep acquiring new skills and knowledge without compromising its existing capabilities. Parametric methods suffer from catastrophic forgetting and loss of plasticity. Non-parametric methods, such as prompts, skills, and harnesses, leverage in-context learning but are limited by context length and lossy plain-text compression. To effectively combine their complementary strengths, we propose HERD (Harness-Evolving Recursive Distillation), a hybrid continual learning framework that learns efficiently in the harness and internalizes experience into cumulative capabilities via distillation. On the harness side, HERD continually refines its skills to capture reusable patterns, while retaining high-residual examples that the skills fail to capture in a bounded memory. On the parameter side, HERD performs drift-constrained distillation with dual anchors, substantially reducing forgetting over long recursive runs. On the 25-stage task stream of TRACE+17, under fixed memory and compute budgets, HERD matches the strongest parametric baseline in new-task acquisition while exhibiting the least forgetting and the strongest retention of general capabilities.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Forgetting, Leverage (statistics), Computer science, Machine learning, Context (archaeology), Artificial intelligence

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange