Newest papers

Did the Problem Actually Get Harder? Testing LLMs on Paired P and NP-Complete Problems Hardened by Adversarial Multi-Agent Rewriting

Nurul Labib Sayeedi, Md. Faiyaz Abdullah Sayeedi, Zabir Al Nazi, Mahbub E. Sobhani, Shubhashis Roy Dipta · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30103 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

多くのNP-complete(NPC)問題には、単一の構造細部で異なるポリノミアルタイム(P)双子がある、例えば3対2カラーグラフ。理由ではなく回想を反映するモデルは、この境界のどの側面に問題が落ちるかを注意しなければならないが、既存のベンチマークは、それぞれのタスクの文法を修正するか、その複雑さのクラスを無視するので、彼らは両方を分離することはできない。我々は、12の問題ファミリーのベンチマークであるP2NPCBenchを導入し、各NPC問題が同じ表面上に置かれたP双子と並行されるとともに実行可能なテキストゲートが並んでいます。見られる文書のサインを取り除く

Machine translation (Meta M2M100); the English original follows.

Abstract

Many NP-complete (NPC) problems have a polynomial-time (P) twin that differs in a single structural detail, such as 3- versus 2-colouring a graph. A model that reasons rather than recalls should notice which side of this boundary a problem falls on, yet existing benchmarks either fix the wording of each task or ignore its complexity class, so they cannot separate the two. We introduce P2NPCBench, a benchmark of 12 problem families in which every NPC problem is paired with a P twin posed over the same surface. To remove the textbook cues that enable recall, we build it with AgentGAN, an adversarial multi-agent framework in which a generator rewrites each statement until a panel of frozen code models fails, while an executable validity gate keeps every rewrite answer- and class-preserving. The result is 2,114 hardened statements with hand-verified reference human solvers. Evaluating 12 proprietary and open-source models, we find that every model scores lower on NPC statements than on their P twins (35.4% vs. 63.8% pass@1), and that 95.1% of model-family pairs fail more often on the NPC side. A tenfold time limit raises NPC accuracy only from 28.5% to 31.9%, so failures stem from wrong or crashing programs rather than slow ones. Hardening also narrows the class gap from 24.8 to 14.6 points, showing that part of the apparent gap reflects familiarity with textbook phrasing rather than complexity class.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Adversarial system, Notice, Rewriting, Executable, Task (project management), Computer science

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange