Newest papers

Forty Questions: Identifying a Language Model from One-Word Answers

Tapan S. Parikh · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28281 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Sebuah model di belakang API dapat diidentifikasi dari jawaban satu kata untuk pertanyaan sehari-hari seperti "Name a sound" atau "Name a poet". Profil model mencatat seberapa sering ia memberikan setiap jawaban untuk setiap pertanyaan. Kami menetapkan jawaban baru untuk model yang paling mungkin telah memberikan mereka. Pada panel dari 94 model, 40 jawaban mengidentifikasi model yang tepat 92.5% dari waktu, melawan 1.1% secara acak. Profile yang dibangun melalui OpenRouter mengidentifikasi semua sepuluh model yang dilayani oleh DeepInfra di bawah label buta. Sebuah sistem prompt menurunkan akurasi dengan 6 hingga 16 poin. Kami menerapkan metode ini ke 96 titik akhir dari 13 model terbuka di OpenRouter, dan 92 mengidentifikasi sebagai model yang mereka klaim. Dari empat lainnya, satu cocok dengan model yang diklaim dengan alasan publik dimatikan. Satu melayani Gemma 4 di bawah nama L

Machine translation (Meta M2M100); the English original follows.

Abstract

A model behind an API can be identified from its one-word answers to everyday questions such as "Name a sound" or "Name a poet". A model's profile records how often it gives each answer to each question. We attribute new answers to the model most likely to have given them. On a panel of 94 models, 40 answers identify the exact model 92.5% of the time, against 1.1% by chance. Profiles built through OpenRouter identified all ten models served by DeepInfra under blinded labels. A system prompt lowers accuracy by 6 to 16 points. We applied the method to 96 endpoints of 13 open models on OpenRouter, and 92 identify as the model they claim. Of the other four, one matches the claimed model with reasoning turned off. One serves Gemma 4 under the name Llama 4 Maverick: it names itself Gemma 4, and it is billed Gemma 4's token counts on every message. Two remain unresolved. Billing records also show context added to first-party models. Grok 4.7 is billed 1,247 input tokens for a one-line question that costs GPT-4o only 12 tokens, and five models state the current date without being billed for it. The questions come from a public census of model defaults, and the method costs cents per endpoint.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Context (archaeology), Computer science, Security token, Natural language processing, Artificial intelligence, Language model

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange