Newest papers

From Prose to Properties: Testing Research Claims with ARATest for Agent-Native Science

Yifeng He, Jiachen Liu, Hao Chen · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30495 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

AI検証エージェントは現在、適切な検証結果を確認し、検証結果を確認し、検証エージェントは検証することができるよりも早く検証を生成します。これらの実験的主張は、そのケースを侵害する必要がある財産を意味しますが、それはプロソーにのみ属性を示すことができます。明示的な属性なしで、検証は、どの証拠が主張を偽造するかを判断することができず、検証結果は、検証エージェントが生成するものよりも速く証拠を生成します。主張を検証するために、我々は、主張の証拠の比較量、範囲、および数値のポリシーを抽出することによって証拠のセマンティクスを修正します。共有ケースのタイプは、主張の統

Machine translation (Meta M2M100); the English original follows.

Abstract

AI agents now run experiments and report findings, so research produces claims faster than researchers can check them. Each empirical claim implies a property that must hold over its cases, but it states that property only in prose. Without an explicit property, no test can decide which evidence would falsify a claim, so validation cannot keep pace with the claims agents produce. To make claims testable, we approximate a claim's semantics by abstracting its property skeleton, a parameterized relation that recurs across claims. We present ARATest, a property-based testing library that checks these skeletons as executable tests. A typed declaration binds a claim to a skeleton and fixes the claim's compared quantities, cases, scope, and numerical policy. A shared case type makes every generator supply cases in the claim's statistical unit. A three-valued semantics assigns each property one of three verdicts: holds, violated, or inconclusive. Every violation carries a replayable witness. A default generator reads the evidence that agent-native research artifacts (ARA) link to each claim, and auto-research generators dispatch an agent to run released code for fresh cases. Our DissClaimer plugin supplies that agent and runs these generators as continuous testing. Of 243 claims in 33 ARA artifacts, 54 received faithful declarations bound to evidence: 32 held, eight were violated, and 14 remained inconclusive. Two violations exposed errors in published papers that their authors confirmed. ARATest detected all 553 seeded faults without a false alarm on 19 valid controls, outperforming two LLM judges. Fresh runs exposed behaviors that reported evidence could not show. These results suggest that one fixed test can check explicit claim properties against both reported and fresh evidence.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Executable, Property (philosophy), Generator (circuit theory), Computer science, Parameterized complexity, Programming language

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange