Newest papers

Beyond 4 bits: Data-Centric Post-training Quantization and Recovery

Rhea Sanjay Sukthanker, Riccardo Grazzi, James J. Hensman, Pashmina Cameron · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28537 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

トレーニング後の量子化(PTQ)は、制限されたコンピューティングとデータで訓練されたモデルの重量を圧縮します。 4 ビットまで、それはほぼ無損ですが、 2 ビットのスカラー PTQ 構成はチャンスレベルの正確さに落ちます。我々は、通常、4 ビットでわずかに重要な実装細部として扱われた3 つの選択を圧縮しますが、2 ビットの品質を決定します:量子化の代表性、効果的なカリブレーションデータ、および復旧の細部調節データは、6 つの密集したおよび専門家のデータの混合モデル(3.8 $から $32 $Bのパラメータにまで)を含みます。 代表性は最大の要因であり、ベクトルコード

Machine translation (Meta M2M100); the English original follows.

Abstract

Post-training quantization (PTQ) compresses the weights of a trained model with limited compute and data. Down to 4 bits it is near-lossless, but at 2 bits scalar PTQ configurations fall to chance-level accuracy. We study three choices usually treated as implementation details that matter little at 4 bits but decide 2-bit quality: the quantization representation, the calibration data, and the recovery fine-tuning data, across six dense and mixture-of experts models (ranging from $3.8$ to $32$B parameters). Representation is the largest factor and vector codebooks restore usable models where scalar PTQ pipelines collapse. Most of the remaining loss in accuracy is on \emph{generative} tasks, and the amount of performance drop is model-dependent. Furthermore, at a short recovery budget, calibration and recovery data are coupled. The best recovery data depends on the calibration data, so the two must be chosen jointly. The resulting recipe involves a vector codebook representation, a small task-aligned calibration mixture, and an inexpensive codebook-only recovery stage. Including codebooks, it uses about $2.3$ effective bits per weight and retains $89$ to $97\%$ of full-precision question-answering (QA) accuracy where scalar PTQ is at chance, and, including recovery, uses $1.4$ to $30\times$ less compute than the strongest quantization-aware-training (QAT) baseline.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Codebook, Quantization (signal processing), USable, Algorithm, Mathematics, Vector quantization

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange