Newest papers

Forty Questions: Identifying a Language Model from One-Word Answers

Tapan S. Parikh · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28281 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Un modelo detrás de una API puede ser identificado a partir de sus respuestas de una palabra a preguntas cotidianas como "Name un sonido" o "Name un poeta". El perfil de un modelo registra cuántas veces da cada respuesta a cada pregunta. Atribuimos nuevas respuestas al modelo más probable de haberlas dado. En un panel de 94 modelos, 40 respuestas identifican el modelo exacto 92.5% del tiempo, frente al 1.1% por casualidad. De los otros cuatro, los perfiles construidos a través de OpenRouter identificaron a todos los diez modelos servidos por DeepInfra bajo etiquetas ciegas. Un sistema prompt baja la precisión de 6 a 16 puntos. Aplicamos el método a 96 puntos finales de 13 modelos abiertos en OpenRouter, y 92 se identifican como el modelo que reclaman. De los otros cuatro, uno coincide con el modelo

Machine translation (Meta M2M100); the English original follows.

Abstract

A model behind an API can be identified from its one-word answers to everyday questions such as "Name a sound" or "Name a poet". A model's profile records how often it gives each answer to each question. We attribute new answers to the model most likely to have given them. On a panel of 94 models, 40 answers identify the exact model 92.5% of the time, against 1.1% by chance. Profiles built through OpenRouter identified all ten models served by DeepInfra under blinded labels. A system prompt lowers accuracy by 6 to 16 points. We applied the method to 96 endpoints of 13 open models on OpenRouter, and 92 identify as the model they claim. Of the other four, one matches the claimed model with reasoning turned off. One serves Gemma 4 under the name Llama 4 Maverick: it names itself Gemma 4, and it is billed Gemma 4's token counts on every message. Two remain unresolved. Billing records also show context added to first-party models. Grok 4.7 is billed 1,247 input tokens for a one-line question that costs GPT-4o only 12 tokens, and five models state the current date without being billed for it. The questions come from a public census of model defaults, and the method costs cents per endpoint.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Context (archaeology), Computer science, Security token, Natural language processing, Artificial intelligence, Language model

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange