Newest papers

From Candidate Admission to Task-Level Return: Auditing LLM-Generated Code under Execution Contracts

Hiroyasu Usami, Keisuke Hara, Ayato Tsuboi, Naohiko Matsuda · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.24609 · cited by 0 works (OpenAlex) · open-access PDF

German

Die Zulassungsregeln behalten generierte Programme, während die Auswahlrichtlinien einen Kandidaten pro Aufgabe zurückgeben. Wir prüfen, wie diese Entscheidungen mit Ausführungsverträgen in zwei getrennt eingefrorenen, disjoint HumanEvalPlus-Kohorten interagieren, die jeweils 32 Aufgaben, vier Proben pro Aufgabe pro Profil und zwei servierte Profile enthalten. Im Rahmen eines Inline-Ausführungsvertrags erlaubt ein struktureller und selbstprüfender Bildschirm 4-9 erweiterte Suite-Nonpasses pro 80-107 Zulassungen und schließt 14-43 Suite-Passes pro 128 geplante Versuche aus. Eine begrenzte Modul-Lieferbedingung fügt 20 und 14 passende Zulassungen für ein Profil in den jeweiligen Kohorten hinzu, plus einen Follow-up-

Machine translation (Meta M2M100); the English original follows.

Abstract

Admission rules retain generated programs, whereas selection policies return one candidate per task. We audit how these decisions interact with execution contracts in two separately frozen, disjoint HumanEvalPlus cohorts, each comprising 32 tasks, four samples per task per profile and two served profiles. Under the inline execution contract, a structural-and-self-test screen admits 4-9 expanded-suite nonpasses per 80-107 admissions and excludes 14-43 suite passes per 128 planned attempts. A restricted module-supply condition adds 20 and 14 passing admissions for one profile in the respective cohorts, plus one follow-up nonpass, and reduces the excluded suite passes to 14-23. All fixed first-admitted outcomes and abstentions remain unchanged. In the original pool, the added passes enter already all-passing admitted pools; in the follow-up, four of 24 common sample orders lose one passed task. Selected-subset diagnostics separately address registration-sensitive completion and reference-relative assertion disagreement, accounting for input domains and comparator tolerances. In these high-pass pools, candidate retention and task-level return diverge and are reported separately. Results are conditional on the saved pools, contracts and pinned suites; downstream training effects are unmeasured.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Audit, Suite, Task (project management), Computer science, Disjoint sets, Sample (material)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange