Newest papers

Beyond 4 bits: Data-Centric Post-training Quantization and Recovery

Rhea Sanjay Sukthanker, Riccardo Grazzi, James J. Hensman, Pashmina Cameron · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28537 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

A quantização pós-treinamento (PTQ) comprime os pesos de um modelo treinado com computação e dados limitados. Até 4 bits é quase sem perdas, mas em 2 bits as configurações escalares do PTQ caem para a precisão de nível de chance. Estudamos três escolhas geralmente tratadas como detalhes de implementação que importam pouco em 4 bits, mas decidem sobre a qualidade de 2 bits: a representação de quantização, os dados de calibração eficazes e os dados de ajuste fino de recuperação, em seis modelos de dados densos e misturados de especialistas (que variam de US $ 3.8 a US $ 32 B parâmetros). A representação é o maior fator e os códigos de vetores são os melhores para restaurar modelos usáveis onde os pipelines escalar

Machine translation (Meta M2M100); the English original follows.

Abstract

Post-training quantization (PTQ) compresses the weights of a trained model with limited compute and data. Down to 4 bits it is near-lossless, but at 2 bits scalar PTQ configurations fall to chance-level accuracy. We study three choices usually treated as implementation details that matter little at 4 bits but decide 2-bit quality: the quantization representation, the calibration data, and the recovery fine-tuning data, across six dense and mixture-of experts models (ranging from $3.8$ to $32$B parameters). Representation is the largest factor and vector codebooks restore usable models where scalar PTQ pipelines collapse. Most of the remaining loss in accuracy is on \emph{generative} tasks, and the amount of performance drop is model-dependent. Furthermore, at a short recovery budget, calibration and recovery data are coupled. The best recovery data depends on the calibration data, so the two must be chosen jointly. The resulting recipe involves a vector codebook representation, a small task-aligned calibration mixture, and an inexpensive codebook-only recovery stage. Including codebooks, it uses about $2.3$ effective bits per weight and retains $89$ to $97\%$ of full-precision question-answering (QA) accuracy where scalar PTQ is at chance, and, including recovery, uses $1.4$ to $30\times$ less compute than the strongest quantization-aware-training (QAT) baseline.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Codebook, Quantization (signal processing), USable, Algorithm, Mathematics, Vector quantization

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange