Newest papers

Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation

Gabriele Dominici, Teague McMillan, Fazl Barez, Marc Langheinrich, Martin Gjoreski · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31639 · cited by 0 works (OpenAlex) · open-access PDF

French

Dans la génération autorégressive standard, l'attention dense permet à chaque étape de raisonnement d'assister au contexte complet, permettant au modèle de dériver des réponses à travers une recherche directe tout en générant des justifications plausibles, mais post-hoc. Nous introduisons CausalCoT, un cadre d'inférence qui impose la transparence causale en décomposant les problèmes en un graphique acyclique dirigé de sous-questions atomiques, en évaluant chaque nœud dans un contexte isolé ne contenant que ses parents causaux directs. Cette structure Markovian enlève les voies d'attention au contenu non parental, rendant les prédictions en aval sensibles au raisonnement local plutôt que au contexte local. À travers une série de raisonnements sensibles,

Machine translation (Meta M2M100); the English original follows.

Abstract

Chain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In standard autoregressive generation, dense attention allows each reasoning step to attend to the full context, enabling the model to derive answers through direct lookup while generating plausible-sounding but post-hoc justifications. We introduce CausalCoT, an inference framework that enforces causal transparency by decomposing problems into a directed acyclic graph of atomic sub-questions, evaluating each node in an isolated context containing only its direct causal parents. This Markovian structure removes attention pathways to non-parent content, making downstream predictions sensitive to local reasoning rather than global context leakage. Across a diverse suite of reasoning benchmarks, CausalCoT preserves competitive predictive performance while improving explanation faithfulness, increasing causal sensitivity of intermediate reasoning steps by an average of 12 and up to 21 percentage points over the strongest baseline. The structured dependency graph additionally enables test-time interventions in which local corrections propagate predictably through downstream reasoning.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Inference, Causal inference, Context (archaeology), Directed acyclic graph, Causal chain

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange