Newest papers

Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination

Miguel Fernandez-de-Retana, Unai Zulaika, Ruben Sanchez-Corcuera, Aritz Bilbao-Jayo, Aitor Almeida · 2026 · arXiv

DOI: 10.48550/arXiv.2610.31586 · open-access PDF

French

L'exactitude des examens médicaux est une preuve courante offerte pour l'utilisation clinique des modèles de langue, mais un système de santé qui entend exécuter un modèle ouvert sur son propre matériel a besoin de savoir beaucoup plus: si le modèle reconnaît quand il est susceptible d'être faux, si le fine-tuning médical améliore cette capacité, si ses réponses correctes reposent sur un raisonnement clinique solide, ou si il utilise réellement les images qu'il donne.

Machine translation (Meta M2M100); the English original follows.

Abstract

Accuracy on medical examinations is common evidence offered for the clinical use of language models, yet a health system that intends to run an open model on its own hardware needs to know considerably more: whether the model recognizes when it is likely to be wrong, whether medical fine-tuning improves this ability, whether its correct answers rest on sound clinical reasoning, or whether it actually uses the images it is given. We examine these questions for four open models of 8 to 27 billion parameters on the Spanish MIR residency examination, scoring 1,552 questions from eight editions und

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange