Julio C. Amador Diaz Lopez · 2026 · arXiv
DOI: 10.48550/arXiv.2610.26601 · open-access PDF
Los modelos de lenguaje enmascarado (MLMs) como BERT se utilizan típicamente para el aprendizaje de representación a pesar de su pretraining denoisante. Nosotros probamos si esta capacidad soporta la generación condicional sin formación de difusión. Nuestro decodificador predijo repetidamente tokens enmascarados y se compromete a predicciones de alta confianza utilizando un horario cosino. No requiere tiempo de difusión ni objetivo de difusión inversa. Frozen BERT ya muestra una capacidad de denosis sustancial, reconstruyendo 61.0%, 37.5%, y 24.4% de los tokens enmascarados en 15%, 45%, y 60% de corrupción, respectivamente, aunque la generación deteriora de forma aguda.
Machine translation (Meta M2M100); the English original follows.
Masked language models (MLMs) such as BERT are typically used for representation learning despite their denoising pretraining. We test whether this capability supports conditional generation without diffusion training. Our decoder repeatedly predicts masked tokens and commits high-confidence predictions using a cosine schedule. It requires no diffusion timestep or reverse-diffusion objective. Frozen BERT already exhibits substantial denoising capability, reconstructing 61.0%, 37.5%, and 24.4% of masked tokens at 15%, 45%, and 60% corruption, respectively, although generation deteriorates sharp
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26601, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange