Newest papers

When Code Gets Cheap, Evaluation Gets Scarce: An Experience Report on AI-Assisted Agile Software Engineering

Barbara Weber, Olaf Zimmermann, Marco Franceschetti · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30916 · cited by 0 works (OpenAlex) · open-access PDF

Korean

이 코딩 도어 실행 도메인은 코드 생산을 가속화하는 데 도움을 주고 있습니다. 대부분의 계정은 그곳에서 중지합니다. 우리는 코딩 도어 실행 기록을 가속화하는 데 도움을 주고 있습니다. 우리는 코딩 도어 실행 도어를 가속화하는 데 도움을 주고 있습니다. 우리는 코딩 도어를 가속화하는 데 도움을 주고 있습니다. 대부분의 계정은 최초의 시스템은 3 일 만에 구축되었으며 실제 코호트가 될 수 있습니다. 우리는이 경험에서 AI가 소프트웨어 엔지니어링을 덜 빠르게 프로그래밍을 변경함으로써 평가합니다. 코드 작성은 손으로 코드를 많이 만듭니다. 신뢰할 수있는 평가 프로세스가 stakeholder persona에서 감사 된 행동에 이르기 때문에 결과가 올바르

Machine translation (Meta M2M100); the English original follows.

Abstract

Coding assistants are widely reported to speed up code production; most accounts stop there. We report the end-to-end construction and deployment of a compliance-critical, human-in-the-loop exam-grading system, built by a single developer who wrote no code by hand, protected by a gated process from stakeholder persona to audited behavior. The initial system was built in three working days and could grade a real cohort. We draw one claim from this experience: AI changes software engineering less by making programming faster than by changing what is scarce. Writing code becomes abundant. Reliable evaluation-knowing whether a result is correct and meets the actual need-remains necessary at every iteration and becomes the binding constraint. Engineering judgment remains a second scarcity. Several defects passed the test suite: they sat at the boundaries the tests had mocked, the browser and the language-model API, or appeared only in real use. Deployment exposed assumptions that local tests had missed. Moreover, an AI assistant acting across many steps bypassed process checkpoints until we made them enforceable gates. All of this was observed in a triple union: the developer was also the end user and domain expert, making expert judgment immediately available and evaluation as cheap as it can be. This favors the iteration gains, which may shrink where evaluation requires additional people or expertise, but makes the case a hard test for evaluation scarcity: evaluation still constrained progress. We draw five lessons: run a gated, traceable construction process; specify what is knowable and discover the rest against a running prototype; validate in the real environment; gate the assistant, not only the work; record decisions with their assumptions and the process state the assistant forgets. A second project showed similar patterns; an independent single-expert case corroborates the shift.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Software engineering, Agile software development, Computer science, Software deployment, Software inspection, Process (computing)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange