Newest papers

DIAL: Steerable Language Models with Distilled Instruction-Adherence Levels

Thanh Q. Tran, John Russell Himawan, Arun Verma, Kiwan Wong, Wei Xiao, Daniela Rus, Bryan Kian Hsiang Low · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31438 · cited by 0 works (OpenAlex) · open-access PDF

French

Lorsqu'une instruction Stepting est un contrôle partagé par des familles de personnes, la réponse de la LLM est-elle courte? Plus généralement, les utilisateurs et les applications nécessitent différents niveaux d'adhésion à l'instruction, mais le prompting ne fournit pas de manière fiable un contrôle de grain fin sur eux. Nous étudions par conséquent des modèles de langues compétitives « Steerable » qui fournissent un contrôle numérique de grain fin sur le degré dans lequel les modèles suivent les instructions générales spécifiées dans le texte. Cependant, les approches existantes n'offrent pas conjointement un contrôle de grain fin, une efficacité et une évolutivité sur les différentes instructions. Pour combler ce fossé, nous introduisons

Machine translation (Meta M2M100); the English original follows.

Abstract

When a user says "Keep it short", how short should the LLM's response be? More generally, users and applications require different levels of instruction adherence, but prompting does not reliably provide fine-grained control over them. We therefore study "steerable language models" that provide fine-grained numerical control over the degree to which models follow general instructions specified in text. However, existing approaches do not jointly offer fine-grained control, efficiency, and scalability across diverse instructions. To address this gap, we introduce DIAL (Distilled Instruction-Adherence Levels) to achieve all three properties. For fine-grained control, we perform classifier-free guidance (CFG) with a frozen LLM to provide training signals for on-policy distillation of instruction adherence at different levels. For efficiency, we design a lightweight steering adapter that uses a classifier over the LLM's hidden states to compute a closed-form steering edit. The classifier adaptively sets the steering magnitude at each token based on the target adherence level, while accounting for the geometry of the LLM's hidden states to preserve response coherence. For scalability, the classifier is text-conditioned, allowing a shared adapter to support new instructions without per-instruction fitting, and training requires only instruction and request text. Evaluation across multiple LLM families shows that DIAL achieves a competitive balance between concept expression and response fluency, with concept scores increasing monotonically across adherence levels. We introduce SteerIF, an instruction-only dataset of 98, 625 concept, persona, verifiable, and descriptive instructions. When trained on SteerIF, DIAL provides fine-grained control over personas and, when combined with prompting, improves verifiable instruction following.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Classifier (UML), Security token, Language model, Artificial intelligence, Scalability

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange