Newest papers

From Candidate Admission to Task-Level Return: Auditing LLM-Generated Code under Execution Contracts

Hiroyasu Usami, Keisuke Hara, Ayato Tsuboi, Naohiko Matsuda · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.24609 · cited by 0 works (OpenAlex) · open-access PDF

French

Les règles d'admission conservent les programmes générés, tandis que les politiques de sélection renvoient un candidat par tâche. Nous vérifions comment ces décisions interagissent avec les contrats d'exécution dans deux cohortes HumanEvalPlus séparément gelées et disjointes, chacune comprenant 32 tâches, quatre échantillons par tâche par profil et deux profils servis. Sous un contrat d'exécution en ligne, un écran de suivi et d'auto-test admet 4-9 passages de suite élargis par 80-107 admissions et exclut 14-43 passages de suite par 128 tentatives planifiées. Dans le pool d'origine, les passages de module restreints ajoutent 20 et 14 passages pour un profil dans les cohortes respectives, plus

Machine translation (Meta M2M100); the English original follows.

Abstract

Admission rules retain generated programs, whereas selection policies return one candidate per task. We audit how these decisions interact with execution contracts in two separately frozen, disjoint HumanEvalPlus cohorts, each comprising 32 tasks, four samples per task per profile and two served profiles. Under the inline execution contract, a structural-and-self-test screen admits 4-9 expanded-suite nonpasses per 80-107 admissions and excludes 14-43 suite passes per 128 planned attempts. A restricted module-supply condition adds 20 and 14 passing admissions for one profile in the respective cohorts, plus one follow-up nonpass, and reduces the excluded suite passes to 14-23. All fixed first-admitted outcomes and abstentions remain unchanged. In the original pool, the added passes enter already all-passing admitted pools; in the follow-up, four of 24 common sample orders lose one passed task. Selected-subset diagnostics separately address registration-sensitive completion and reference-relative assertion disagreement, accounting for input domains and comparator tolerances. In these high-pass pools, candidate retention and task-level return diverge and are reported separately. Results are conditional on the saved pools, contracts and pinned suites; downstream training effects are unmeasured.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Audit, Suite, Task (project management), Computer science, Disjoint sets, Sample (material)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange