Newest papers

Forty Questions: Identifying a Language Model from One-Word Answers

Tapan S. Parikh · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28281 · cited by 0 works (OpenAlex) · open-access PDF

German

Ein Modell hinter einer API kann aus seinen Ein-Wort-Antworten auf alltägliche Fragen wie "Name a Sound" oder "Name a Poet" identifiziert werden. Das Profil eines Modells zeichnet auf, wie oft es jede Antwort auf jede Frage gibt. Wir geben dem Modell neue Antworten zu. Auf einem Panel von 94 Modellen identifizieren 40 Antworten das genaue Modell 92.5% der Zeit, gegen 1.1% zufällig. Profile, die über OpenRouter erstellt wurden, identifizierten alle zehn Modelle, die von DeepInfra unter blinded Labels serviert wurden. Ein System-Prompt senkt die Genauigkeit um 6 bis 16 Punkte. Wir haben die Methode auf 96 Endpunkte von 13 offenen Modellen auf OpenRouter angewendet, und 92 identifizieren als das Modell, das sie behaupten. Von den anderen vier

Machine translation (Meta M2M100); the English original follows.

Abstract

A model behind an API can be identified from its one-word answers to everyday questions such as "Name a sound" or "Name a poet". A model's profile records how often it gives each answer to each question. We attribute new answers to the model most likely to have given them. On a panel of 94 models, 40 answers identify the exact model 92.5% of the time, against 1.1% by chance. Profiles built through OpenRouter identified all ten models served by DeepInfra under blinded labels. A system prompt lowers accuracy by 6 to 16 points. We applied the method to 96 endpoints of 13 open models on OpenRouter, and 92 identify as the model they claim. Of the other four, one matches the claimed model with reasoning turned off. One serves Gemma 4 under the name Llama 4 Maverick: it names itself Gemma 4, and it is billed Gemma 4's token counts on every message. Two remain unresolved. Billing records also show context added to first-party models. Grok 4.7 is billed 1,247 input tokens for a one-line question that costs GPT-4o only 12 tokens, and five models state the current date without being billed for it. The questions come from a public census of model defaults, and the method costs cents per endpoint.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Context (archaeology), Computer science, Security token, Natural language processing, Artificial intelligence, Language model

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange