Newest papers

Mixing Matters: Evidence Position Bias Across Sequence Mixers in Long-Context Question Answering

Aman Behera, Namit Solanki, Mehul Anand · 2026 · arXiv

DOI: 10.48550/arXiv.2610.31135 · open-access PDF

Portuguese

Os modelos de contexto longo frequentemente usam evidências de resposta menos eficazes quando aparecem longe de um limite imediato. Comparamos esse viés de posição entre as arquiteturas de Transformer, estado-espaço e Mamba híbrido movendo um documento de ouro através de um conjunto fixo de nove distractores. Os Transformers avaliados mostram uma vantagem inicial que os modelos de estado-espaço avaliados carecem, enquanto ambas as famílias favorecem evidências perto do fim. Uma comparação de Mamba pura-versus-híbrida correspondente aponta na mesma direção, mas permanece estatisticamente incerta.

Machine translation (Meta M2M100); the English original follows.

Abstract

Long-context models often use answer-bearing evidence less effectively when it appears away from a prompt boundary. We compare this position bias across Transformer, state-space, and hybrid Mamba architectures by moving one gold document through a fixed set of nine distractors. The evaluated Transformers show a beginning advantage that the evaluated state-space models lack, while both families favor evidence near the end. A matched pure-versus-hybrid Mamba comparison points in the same direction but remains statistically uncertain. The family difference appears among models that can answer the

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange