Newest papers

Agent Policy-Value Audit: Separating Transition Composition from Event Selection in Financial LLM Agents

Mingyang, Chen, Yida, Xu, Huiwen, Chen, Yiming Lu, Wei Jin · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.04040 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

金融代理人LLM代理人经常被评估,将其端到端的回报与基线的回报进行比较,并对配对差异对零进行测试。这衡量了是否部署代理人改变了实现的绩效,但它不孤立事件选择技巧。经常从平板到长的位置改变的代理人可以从上游事件池中获得积极的对等回报,即使在随机选择事件时也产生了差异。我们提出了代理人政策价值审计,该审计将每个订购的行动变化类型的观察数固定下来,并随机将其重新分配到符合条件的事件中。从这些重新分配中获得的平均回报是组成基准;观察到的部署值与此基准的选择值之间的差异是半合成基准值。在基于实际收益回报的

Machine translation (Meta M2M100); the English original follows.

Abstract

Financial LLM agents are often evaluated by comparing their end-to-end returns with those of a baseline and testing the paired difference against zero. This measures whether deploying the agent changes realized performance, but it does not isolate event-selection skill. An agent that frequently changes positions from flat to long can earn a positive paired return from an upward-drifting event pool even when it selects events at random. We propose the Agent Policy-Value Audit, which holds fixed the observed count of each ordered action-change type and randomly reassigns them across eligible events. The average payoff from these reassignments is the composition benchmark; the difference between observed deployment value and this benchmark is selection value. In semi-synthetic benchmarks based on real earnings-event returns, a zero-centered paired test falsely attributes passive exposure to selection skill in $11.6\%$ of no-skill replications, while the transition-matched audit reduces this rate to $5.3\%$. Applied retrospectively to 723 earnings events at 44 U.S. consumer-facing firms, the audit decomposes the agent's gross deployment value of $+15.2$ bps/event into a $+25.8$ composition benchmark and a $-10.6$ selection value. The agent does not detectably outperform matched random assignments. Financial-agent evaluations should report deployment value separately from event-selection value.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Benchmark (surveying), Stochastic game, Earnings, Selection (genetic algorithm), Audit, Event (particle physics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange