Daniel Robert Kling Alexander, Catherine Louise Kling · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.10506 · cited by 0 works (OpenAlex) · open-access PDF
现在对大型语言模型提出的许多问题没有正确的答案:政策值是什么,用户应该选择哪种选项,如何衡量竞争价值。声明偏好经济学几十年来一直面临着这个问题。它评估调查响应而不了解真正的价值,通过有效性框架和相关概念:内容,构建和标准有效性,可靠性,激励兼容性和后果性。我们认为这个框架是评估语言模型的一般方法,我们阐述了每个概念对LLM评估意味着什么。我们通过使用发布的水质量声明偏好经济评估调查(Vossler et al. 2023)展示了方法,以六种模型进行管理。在这个经济应用中,有效性测试以经济学的预测的形式进行:需求应该下降,并愿意支付应对良好和收入的范围。对测试的正确性
Machine translation (Meta M2M100); the English original follows.
Many of the questions now put to large language models have no correct answer to score against: what a policy is worth, which option a user should choose, how to weigh competing values. Stated-preference economics has faced this problem for decades. It judges survey responses without knowing the true value, through a framework of validity and related concepts: content, construct, and criterion validity, reliability, incentive compatibility, and consequentiality. We argue that this framework is a general method for evaluating language models, and we set out what each concept means for LLM evaluation. We demonstrate the approach using a published water-quality stated preference economic valuation survey (Vossler et al. 2023) administered to six models. In this economic application, the validity tests take the form of predictions from economic theory: demand should slope down, and willingness to pay should respond to the scope of the good and to income. The tests separate the models sharply. Two older models fail the most basic test at a household income level of \$75,000, and the two newest pass every test of theoretical validity we can score, but diverge on convergent validity. Passing validity tests shows that a model's answers are coherent, not that they are correct.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Valuation (finance), Test (biology), Set (abstract data type), Incentive, Economics, Contingent valuation
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.10506, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange