Newest papers

Validating Autoformalized Specifications via Contrapuntal Languages

Yihan Dai, Xuanlin Liu, Haoxiang Jia, Tianchi Li, Xin Zhang, Sergey Mechtaev · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30059 · cited by 0 works (OpenAlex) · open-access PDF

Korean

공식적인 기본 사양은 AI 생성 된 코드를 기계적으로 검사 할 수 있습니다. 사용자 의도는 일반적으로 공식 논리 대신 자연 언어로 표현되기 때문에, 사양은 먼저 완전한 프로그램을 통해, 예를 들어, LLM 기반의 구체적인 자동 형식화%를 통해 추출해야합니다. 그러나 자동 형식화 된 사양 자체는 신뢰할 수 없으므로 핵심적인 도전은 사용자의 의도를 인용하지 않고도 비용이 많은 오라클에 의존하지 않는지 검증하는 것입니다. 이 오라클 무료 설정에서, 자연적인 전략은 모델 특정 출력 사이에서 투표해야하지만, 모델 특정 오류가 종종 상호 연관되어 있기 때문에 출력 사이의 합의는 공유 오류를 반영 할 수 있습니다. 우리의 핵심 통찰력은이 오라클의 비용이

Machine translation (Meta M2M100); the English original follows.

Abstract

Formal specifications can make AI-generated code mechanically checkable. Because user intent is typically expressed in natural language rather than formal logic, specifications must first be derived from it, e.g., via LLM-based autoformalization. Yet autoformalized specifications are themselves untrusted, so a key challenge is validating whether they capture user intent without relying on a costly oracle such as human review. In this oracle-free setting, a natural strategy is to vote among model outputs, but because model errors are often correlated, agreement among outputs may reflect shared mistakes rather than correctness. Our key insight is that these errors can be decorrelated through representational diversity. We achieve this diversity with contrapuntal languages: pairs of task-specific formal languages, each designed to express the intended specification, yet admitting no structure-preserving translation between them. An LLM formalizes the intent independently in each language, and we accept a specification only if a solver proves the two formalizations equivalent. To construct such languages, we present a synthesis algorithm that reuses fragments of sampled seed specifications as syntactic sugar over a base specification language. We implement the approach for first-order logic specifications and evaluate it on HumanEval+ and LiveCodeBench using GPT-5.4 mini and Qwen3.6-Flash with and without thinking. Over the strongest baseline, our method increases the proportion of accepted specifications that are correct, i.e., complete and sound, by 12.0% on HumanEval+ and 42.9% on LiveCodeBench. The accepted specifications judged complete reject 98.6% of buggy program variants on HumanEval+ and 96.2% on LiveCodeBench, compared with 92.8% and 78.3% for the strongest baseline.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Programming language, Oracle, Formal specification, Key (lock), Construct (python library)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange