Newest papers

StreamEval: Evaluating Interactive Streaming Video Generation Models

StreamEval Team · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28402 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

ストリーミングビデオの生成が進むにつれて、生成されたビデオとのリアルタイムの相互作用を可能にすることはますます重要になります。StreamEvalは、5つのタスクをカバーするインタラクティブストリーミングビデオ生成のための統一ベンチマークである:アバター対話、アバターパフォーマンス、世界生成、指示編集、参照編集。StreamEvalは、それぞれ15～300秒間のスクリプトテストケースを含み、初期資産を指定し、徐々に到着する多様なリクエスト、および状態変更とコンテンツ保存のための明確な要件を導入します。パフォーマンスは、タスクの実行、状態の一致性、視覚的忠誠度、および

Machine translation (Meta M2M100); the English original follows.

Abstract

As streaming video generation advances, enabling real-time interaction with generated video is becoming increasingly important. We introduce StreamEval, a unified benchmark for interactive streaming video generation spanning five tasks: avatar dialogue, avatar performance, world generation, instruction editing, and reference editing. StreamEval comprises 300 scripted test cases lasting 15-300s, each specifying initial assets, incrementally arriving multimodal requests, and explicit requirements for state changes and content preservation. Performance is assessed along four evaluation dimensions: task execution, state consistency, visual fidelity, and audiovisual coordination, instantiated through ten task-specific metrics on a 0-100 scale. Evaluating 37 distinct models across 300 test cases, we find that high visual quality and identity retention do not necessarily coincide with reliable request execution or sustained state consistency. In particular, request execution remains challenging: the highest task-specific scores are 36.90 for Following among seven avatar-performance models, 25.93 for Following in text-conditioned world generation, and 15.00 for Edit Success among three reported reference-editing models. These findings identify reliable request execution under timing and preservation constraints as a central challenge for advancing omni-modal real-time interactive systems. The leaderboard is available at https://streameval.ai

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Avatar, Benchmark (surveying), Task (project management), State (computer science), Interactive video

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange