Newest papers

Safety Mid-Training: Internalizing LLM Safety as a Foundational Capability

Zhengyue Zhao, Liwei Jiang, Yejin Choi, Chaowei Xiao1 · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26809 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Namun, keamanan dapat menurun ketika pelatihan selanjutnya tidak lagi mencakup data keamanan, dan keuntungan dalam keamanan sering datang pada biaya kemampuan umum. Ketegangan ini menunjukkan bahwa keamanan dan alasan umum tidak harus diperlakukan sebagai tujuan pelatihan yang terpisah. Sebaliknya, kami melihat keamanan sebagai kemampuan LLM yang dapat dipelajari melalui struktur alasan yang dibagikan dengan tugas-tugas umum. Kedua model membutuhkan model untuk memahami maksud atau tujuan tugas pertengahan, menerapkan pengetahuan yang relevan, dan alasan tentang konsekuensi potensial. Kami memperkenalkan keamanan di tengah-tengah pelatihan, antara pelatihan pra-pelatihan dan pelatihan pasca-pelatihan yang secara bersama-sama mengembangkan keamanan dan alasan umum melalui struktur bersama ini. Mengingat keamanan dan utilitas prompts, model dasar menghasilkan dokumen terstruktur dan alasan yang kemudian digunakan untuk pelatihan pertengahan. Pelatihan sub-pelatihan keamanan mengurangi tingkat kesuksesan rata-rata (

Machine translation (Meta M2M100); the English original follows.

Abstract

Safety alignment for large language models (LLMs) is primarily performed during post-training, while recent work has also begun incorporating safety data into pre-training. Yet safety can degrade when later training no longer includes safety data, and gains in safety often come at the cost of general capability. This tension suggests that safety and general reasoning should not be treated as separate training goals. We instead view safety as an LLM capability that can be learned through a reasoning structure shared with general tasks. Both require the model to understand intent or task goals, apply relevant knowledge, and reason about potential consequences. We introduce safety mid-training, a stage between pre-training and post-training that jointly develops safety and general reasoning through this shared structure. Given safety and utility prompts, the base model generates structured documents and reasoning traces that are then used for mid-training. Safety mid-training reduces average attack success rate (ASR) by 72% to 81% relative to Qwen3-1.7B, 4B, and 8B base models, while also improving general capability. The benefits persist through subsequent post-training. After general-only SFT, mid-trained models largely retain their safety, achieving lower ASR than the baselines without sacrificing general capability. They also retain safety under general-only GRPO without safety rewards, while subsequent RL reaches a better safety-utility frontier when initialized from the mid-trained models. Together, these results suggest that learning safety and general reasoning within a shared mid-training stage can reduce the safety-utility trade-off and make safety more robust to subsequent training.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Task (project management), Computer science, Safety behaviors, Risk analysis (engineering), Work (physics), Knowledge base

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange