Newest papers

From Candidate Admission to Task-Level Return: Auditing LLM-Generated Code under Execution Contracts

Hiroyasu Usami, Keisuke Hara, Ayato Tsuboi, Naohiko Matsuda · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.24609 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

As regras de admissão retêm programas gerados, enquanto as políticas de seleção devolvem um candidato por tarefa. Auditamos como essas decisões interagem com contratos de execução em duas coortes separadamente congeladas, dissociadas HumanEvalPlus, cada uma com 32 tarefas, quatro amostras por tarefa por perfil e dois perfis servidos. Sob um contrato de execução inline, uma tela estrutural e auto-teste admite 4-9 passes de suíte expandido por 80-107 admissões e exclui 14-43 passes de suíte por 128 tentativas planejadas. Uma condição de fornecimento de módulo restrito adiciona 20 e 14 passes de admissão para um perfil em uma coorte respectiva, além de uma não-passagem de seguimento e reduz as passes de suíte excluídas para 14-23.

Machine translation (Meta M2M100); the English original follows.

Abstract

Admission rules retain generated programs, whereas selection policies return one candidate per task. We audit how these decisions interact with execution contracts in two separately frozen, disjoint HumanEvalPlus cohorts, each comprising 32 tasks, four samples per task per profile and two served profiles. Under the inline execution contract, a structural-and-self-test screen admits 4-9 expanded-suite nonpasses per 80-107 admissions and excludes 14-43 suite passes per 128 planned attempts. A restricted module-supply condition adds 20 and 14 passing admissions for one profile in the respective cohorts, plus one follow-up nonpass, and reduces the excluded suite passes to 14-23. All fixed first-admitted outcomes and abstentions remain unchanged. In the original pool, the added passes enter already all-passing admitted pools; in the follow-up, four of 24 common sample orders lose one passed task. Selected-subset diagnostics separately address registration-sensitive completion and reference-relative assertion disagreement, accounting for input domains and comparator tolerances. In these high-pass pools, candidate retention and task-level return diverge and are reported separately. Results are conditional on the saved pools, contracts and pinned suites; downstream training effects are unmeasured.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Audit, Suite, Task (project management), Computer science, Disjoint sets, Sample (material)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange