Newest papers

Agent Policy-Value Audit: Separating Transition Composition from Event Selection in Financial LLM Agents

Mingyang, Chen, Yida, Xu, Huiwen, Chen, Yiming Lu, Wei Jin · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.04040 · cited by 0 works (OpenAlex) · open-access PDF

Korean

금융용 LLM 에이전트는 종종 최종적인 수익률과 기본적인 수익률을 비교하여 측정하여 제로 차이를 측정합니다. 이것은 에이전트가 실현된 성과를 변경하는지 여부를 측정하지만 이벤트 선택 기술을 고립하지는 않습니다. 종종 평평한 위치에서 긴 위치를 변경하는 에이전트는 임의로 이벤트를 선택할 때조차 상승하는 이벤트 폴리에서 긍정적 인 쌍방이 된 수익률을 얻을 수 있습니다. 우리는 에이전트 정책 가치 감사를 제안합니다. 이는 각 주문된 액션 변환 유형의 관찰 된 수익률을 고정하고 임의로 합격 가능한 이벤트에 그들을 재분배합니다. 이러한 재분배의 평균 수익률은 구성 기준입니다; 관찰 된 배포 가치와이 기준

Machine translation (Meta M2M100); the English original follows.

Abstract

Financial LLM agents are often evaluated by comparing their end-to-end returns with those of a baseline and testing the paired difference against zero. This measures whether deploying the agent changes realized performance, but it does not isolate event-selection skill. An agent that frequently changes positions from flat to long can earn a positive paired return from an upward-drifting event pool even when it selects events at random. We propose the Agent Policy-Value Audit, which holds fixed the observed count of each ordered action-change type and randomly reassigns them across eligible events. The average payoff from these reassignments is the composition benchmark; the difference between observed deployment value and this benchmark is selection value. In semi-synthetic benchmarks based on real earnings-event returns, a zero-centered paired test falsely attributes passive exposure to selection skill in $11.6\%$ of no-skill replications, while the transition-matched audit reduces this rate to $5.3\%$. Applied retrospectively to 723 earnings events at 44 U.S. consumer-facing firms, the audit decomposes the agent's gross deployment value of $+15.2$ bps/event into a $+25.8$ composition benchmark and a $-10.6$ selection value. The agent does not detectably outperform matched random assignments. Financial-agent evaluations should report deployment value separately from event-selection value.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Benchmark (surveying), Stochastic game, Earnings, Selection (genetic algorithm), Audit, Event (particle physics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange