Daniel Robert Kling Alexander, Catherine Louise Kling · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.10506 · cited by 0 works (OpenAlex) · open-access PDF
Viele der Fragen, die jetzt auf große Sprachmodelle gestellt werden, haben keine korrekte Antwort, um gegen: was eine Politik wert ist, welche Option ein Benutzer wählen sollte, wie man konkurrierende Werte wiegt. Stated-Preference-Ökonomie hat dieses Problem seit Jahrzehnten konfrontiert. Es beurteilt Umfrage Antworten ohne den wahren Wert zu kennen, durch einen Rahmen der Gültigkeit und verwandte Konzepte: Inhalt, Konstruktion und Kriterien Gültigkeit, Zuverlässigkeit, Anreizkompatibilität und Konsequenzalität. Wir argumentieren, dass dieser Rahmen eine allgemeine Methode für die Bewertung von Sprachmodellen ist, und wir legen heraus, was jedes Konzept für die LLM-Bewertung bedeutet. Wir demonstrieren den Ansatz mit einem veröffentlichten Wasser-Qualität erklärt wirtschaftliche Bewertung
Machine translation (Meta M2M100); the English original follows.
Many of the questions now put to large language models have no correct answer to score against: what a policy is worth, which option a user should choose, how to weigh competing values. Stated-preference economics has faced this problem for decades. It judges survey responses without knowing the true value, through a framework of validity and related concepts: content, construct, and criterion validity, reliability, incentive compatibility, and consequentiality. We argue that this framework is a general method for evaluating language models, and we set out what each concept means for LLM evaluation. We demonstrate the approach using a published water-quality stated preference economic valuation survey (Vossler et al. 2023) administered to six models. In this economic application, the validity tests take the form of predictions from economic theory: demand should slope down, and willingness to pay should respond to the scope of the good and to income. The tests separate the models sharply. Two older models fail the most basic test at a household income level of \$75,000, and the two newest pass every test of theoretical validity we can score, but diverge on convergent validity. Passing validity tests shows that a model's answers are coherent, not that they are correct.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Valuation (finance), Test (biology), Set (abstract data type), Incentive, Economics, Contingent valuation
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.10506, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange