Newest papers

SpecMRL: Redesign Matryoshka Representation Learning within Parametric Spectrum

Zhongxin Chen, Songhao Wu, Heng Cui, Cong Li, Xiao Zhou, Rui Yan · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26824 · cited by 0 works (OpenAlex) · open-access PDF

Korean

Matryoshka Representation Learning (MRL)는 결합된 프리픽스를 공동으로 최적화함으로써 유연한 표현 크기를 지원하도록 설계되었습니다. 효과적이지만,이 디자인은 선도 부분이 적절한 컴팩트 인코딩을 제공하도록 명시적인 제한이 부족합니다. 이를 해결하기 위해 우리는 SpecMRL을 제안하여 기능 공간을 재구성하여보다 효과적인 프리픽스 읽기를 가능하게합니다. 구체적으로, 우리는 특징 중요성으로 구성된 파라미터 파생 스펙트럼 기초로 MRL을 확장합니다. SpecMRL은 다른 경우 표준 MRL을 따르며, 프리픽스 읽기가 학습된 스펙트럼 공간에서 수행됩니다. MTEB에 대한 실험은 SpecMRL가 모든 평가된 감소된 차원에서 MRL를

Machine translation (Meta M2M100); the English original follows.

Abstract

Matryoshka Representation Learning (MRL) is designed to support flexible representation sizes by jointly optimizing nested prefixes. While effective, this design lacks an explicit constraint to ensure that the leading portion provides a suitable compact encoding. To address this, we propose SpecMRL, which reorganizes the feature space to enable more effective prefix readout. Specifically, we extend MRL with a parameter-derived spectral basis organized with feature importance. SpecMRL otherwise follows standard MRL, with prefix readout performed in the learned spectral space. Experiments on MTEB show that SpecMRL outperforms MRL at all evaluated reduced dimensions, with larger gains under more aggressive compression. These results suggest that the learned spectral basis better prioritizes informative directions, enabling more effective representation compression. Notably, SpecMRL extends the performance frontier of MRL, surpassing full-dimensional MRL with only half the embedding dimensions.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Representation (politics), Embedding, Computer science, Feature (linguistics), Basis (linear algebra), Feature learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange