Newest papers

SpecMRL: Redesign Matryoshka Representation Learning within Parametric Spectrum

Zhongxin Chen, Songhao Wu, Heng Cui, Cong Li, Xiao Zhou, Rui Yan · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26824 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

Matryoshka Representation Learning (MRL) は、組み込まれたプレフィックスを共同で最適化することにより、柔軟な表示サイズをサポートするように設計されています。効果的であるにもかかわらず、この設計は、主な部分が適切なコンパクトなエンコーディングを提供することを保証するための明示的な制約が欠けています。これに対処するために、我々は、機能スペースを再編成して、より効果的なプレフィックス読み取りを可能にする SpecMRL を提案します。具体的には、我々は、機能の重要性とともに組織されたパラメーター由来のスペクトラルベースで MRL を拡張します。 SpecMRL は、標準的な MRL に従い、学習スペクトラルスペースで実施されたプレフィックス読み取りを

Machine translation (Meta M2M100); the English original follows.

Abstract

Matryoshka Representation Learning (MRL) is designed to support flexible representation sizes by jointly optimizing nested prefixes. While effective, this design lacks an explicit constraint to ensure that the leading portion provides a suitable compact encoding. To address this, we propose SpecMRL, which reorganizes the feature space to enable more effective prefix readout. Specifically, we extend MRL with a parameter-derived spectral basis organized with feature importance. SpecMRL otherwise follows standard MRL, with prefix readout performed in the learned spectral space. Experiments on MTEB show that SpecMRL outperforms MRL at all evaluated reduced dimensions, with larger gains under more aggressive compression. These results suggest that the learned spectral basis better prioritizes informative directions, enabling more effective representation compression. Notably, SpecMRL extends the performance frontier of MRL, surpassing full-dimensional MRL with only half the embedding dimensions.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Representation (politics), Embedding, Computer science, Feature (linguistics), Basis (linear algebra), Feature learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange