Newest papers

MemT-1.0: Towards Memory-Native Evolution

Zhiyu Li, Zeyu Zhang, Linyun Liu, Yiwei Bu, Chenyang Xi, Xichong Zhang, Yi Yu, Yu Yu, Zhichen Liu, Yuxiao Yi, Chunyu Li, Haoran Liu, Zhenkai Ding, Le Zhou, Juncheng Zhang, Boyu Chen, Zehao Lin, Guorui Li, Renyu Fu, Ji Li · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26825 · cited by 0 works (OpenAlex) · open-access PDF

Korean

메모리 계산의 일부로 메모리를 내부화하는 메모리 단계 기반 진화는 기초 모델의 내재적 기능을 만들어 정보를 지속적으로 축적하고 활용할 수 있도록 해줍니다. 이 목적을 위해, 우리는 메모리 계산의 일부로 메모리를 내부화하는 메모리 기초 모델의 새로운 아키텍처인 MemT-1.0을 도입합니다. Metis에 건설된 MemT-1.0은 Gated DeltaNet-2를 사용하여 각 회로 내에서 안정적인 토큰 수준의 메모리 업데이트 메커니즘과 효율적인 훈련 및 추론 인프라를 사용합니다. 우리는 메모리 용량을 확대하기 위해 Mixture-of-Storage를 도입합니다. 의도적인 배경 준비 후, 우리는 메모리 및 일반적인 내부 능력에 대한 포괄적 인

Machine translation (Meta M2M100); the English original follows.

Abstract

Memory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information. To this end, we introduce MemT-1.0, a new architecture of memory foundation models that internalize memory as part of model computation. Building on Metis, MemT-1.0 employs Gated DeltaNet-2 with a stable token-level memory update mechanism within each turn. We further introduce Mixture-of-Storage to scale native memory capacity. Following a deliberate backbone preparation, we develop a two-stage training pipeline consisting of continued pre-training and post-training, supported by 34B tokens of training data and efficient training and inference infrastructure. We release a model family comprising MemT-1.0-Nano and MemT-1.0-Flash, and conduct a comprehensive evaluation of memory and general capabilities. The results show that they outperform the parametric memory baselines on average and remain competitive with retrieval-augmented methods, while largely preserving general capabilities. At a history length of 128K tokens, MemT-1.0-Nano achieves approximately $1.86\times$ end-to-end inference speedup and a $66\times$ reduction in persistent storage relative to full context KV cache. By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Inference, Task (project management), Pipeline (software), Context (archaeology), Memory model

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange