Newest papers

State World Models: Execute the Rules, Generate the Pixels

Chan Yan Ki, Yanran Zhang, Wenzhao Zheng, Xiaofeng Wang, Zheng Zhu, Lei Chen, Jie Zhou, Jiwen Lu · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26811 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

现有世界模型可以从单个图像中生成动作条件的视频。许多人专注于摄像机运动,允许用户探索生成的场景。然而,单独导航并不能确保在其结果取决于当前框架中不可见的状态时,一个动作遵循持续的规则。我们呈现了Stateful Harness,一个框架,将世界进化与视频渲染分开,以支持状态和规则控制的互动。它在规则规定的下保持一个不断演变的世界状态,使每个结果与同一冷冻的视频模型保持一致,并使用生成的剪辑来更新世界状态,以便在下一轮进行更新。在每个轮流中,该剪辑解决了用户选择的动作,并将结果编译成指令,而从可编辑的3D场景

Machine translation (Meta M2M100); the English original follows.

Abstract

Existing world models can generate action-conditioned video from a single image. Many focus on camera movement, allowing users to explore the generated scene. However, navigation alone does not ensure that an action follows persistent rules when its outcome depends on state that is invisible in the current frame. We present Stateful Harness, a framework that separates world evolution from video rendering to support stateful and rule-governed interaction. It maintains an evolving world state under specified rules, renders each resulting event with a frozen video model, and uses the generated clip to update the world state for the next turn. At each turn, the harness resolves the action selected by the user and compiles the resulting event into an instruction, while a preview clip from an editable 3D scene provides spatial and camera guidance. To evaluate rule compliance, we build WorldState-Bench, a benchmark of 50 counterfactual twin pairs that share an initial frame and action but differ in hidden state and required outcome. Under rule-only prompting, the best of the 11 evaluated video backends produces the required endings for both cases in 32% of pairs. For MiniMax-H3, specifying the required endings raises this rate from 32% to 64%, while the full harness reaches 92% with the same frozen model. These results support separating rule execution from video rendering for world interactions with hidden state.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Rendering (computer graphics), Stateful firewall, Counterfactual thinking, Artificial intelligence, Event (particle physics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange