Newest papers

Beyond 4 bits: Data-Centric Post-training Quantization and Recovery

Rhea Sanjay Sukthanker, Riccardo Grazzi, James J. Hensman, Pashmina Cameron · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28537 · cited by 0 works (OpenAlex) · open-access PDF

Russian

Послетренировочная квантизация (PTQ) сжимает весы обученной модели с ограниченными вычислениями и данными. До 4 битов она почти без убытков, но при 2 битах скалярные конфигурации PTQ падают до точности уровня случайности. Мы изучаем три варианта, обычно рассматриваемые как детали реализации, которые имеют малое значение на 4 битах, но решают о качестве 2-битов: квантизация, эффективные данные калибровки и данные восстановления с тонкой настройкой, на шести плотных и смешанных экспертных моделях данных (от $3.8 до $32B параметров). Представление является самым большим фактором и векторные кодеки восстанавливают полезные модели, где скалярные трубопроводы PTQ разрушаются. Большинство оставших

Machine translation (Meta M2M100); the English original follows.

Abstract

Post-training quantization (PTQ) compresses the weights of a trained model with limited compute and data. Down to 4 bits it is near-lossless, but at 2 bits scalar PTQ configurations fall to chance-level accuracy. We study three choices usually treated as implementation details that matter little at 4 bits but decide 2-bit quality: the quantization representation, the calibration data, and the recovery fine-tuning data, across six dense and mixture-of experts models (ranging from $3.8$ to $32$B parameters). Representation is the largest factor and vector codebooks restore usable models where scalar PTQ pipelines collapse. Most of the remaining loss in accuracy is on \emph{generative} tasks, and the amount of performance drop is model-dependent. Furthermore, at a short recovery budget, calibration and recovery data are coupled. The best recovery data depends on the calibration data, so the two must be chosen jointly. The resulting recipe involves a vector codebook representation, a small task-aligned calibration mixture, and an inexpensive codebook-only recovery stage. Including codebooks, it uses about $2.3$ effective bits per weight and retains $89$ to $97\%$ of full-precision question-answering (QA) accuracy where scalar PTQ is at chance, and, including recovery, uses $1.4$ to $30\times$ less compute than the strongest quantization-aware-training (QAT) baseline.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Codebook, Quantization (signal processing), USable, Algorithm, Mathematics, Vector quantization

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange