Newest papers

PsycheMME: A Psychology-Informed Benchmark for Social Understanding and Interaction in Videos

Heran Hu, Yuhang Dong, Zhucun Xue, Kaiwei Chen, Yong Liu, Jiangning Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28625 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

了解人类需求和选择响应需要不同的判断,集合精度可以掩盖。我们引入PsycheMME,一个由心理学提供的视频基准,拥有八种功能,涵盖了基础,状态,需求,互动决策,记忆和风险。利用社交信息处理和感知行动周期,这个组织将人类的理解与响应选择分开。我们通过独立回答的630个事件构建了1000个问题,提出的参考被扣留和专家审查。 我们的理性验证协议通过支持和逻辑一致的明确理性与注明的提示来衡量响应。 在17个模型中,Timing具有最低的验证分和信贷保留,对正确的时间响应的支持判断较弱。 具有相似的需求分数的模型仍然在对开放问题的响应完整性方面

Machine translation (Meta M2M100); the English original follows.

Abstract

Understanding human needs and selecting responses require distinct judgments that aggregate accuracy can obscure. We introduce PsycheMME, a video benchmark informed by psychology, with eight capabilities spanning grounding, states, needs, interaction decisions, memory, and risk. Drawing on Social Information Processing and the Perception-Action Cycle, this organization separates human understanding from response selection. We construct 1,000 questions from 630 episodes through independent answering with proposed references withheld and expert review. Our Reasoning-Verified protocol weights answer credit by the support and logical consistency of explicit rationales against annotated cues. Across 17 models, Timing has the lowest Verified scores and credit retention, with weaker support judgments for correct Timing answers. Models with similar Need scores still differ in answer completeness on open questions. Input controls and SFT/GRPO examine context dependence and adaptation. Evaluating answers alongside rationales locates where models fail across understanding, response selection, and justification.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Benchmark (surveying), Consistency (knowledge bases), Construct (python library), Completeness (order theory), Context (archaeology)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange