Newest papers

Beyond 4 bits: Data-Centric Post-training Quantization and Recovery

Rhea Sanjay Sukthanker, Riccardo Grazzi, James J. Hensman, Pashmina Cameron · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28537 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Setelah pelatihan pengujian kuantitas (PTQ) mengkompresi beban dari model terlatih dengan komputasi dan data terbatas. Hingga 4 bit, itu hampir tanpa kerugian, tetapi pada 2 bit skalar konfigurasi PTQ jatuh ke tingkat kebetulan akurasi. Kami mempelajari tiga pilihan yang biasanya diperlakukan sebagai rincian implementasi yang penting sedikit pada 4 bit tetapi memutuskan kualitas 2 bit: representasi kuantitas, data kalibrasi efektif, dan data pemulihan fin-tuning, di atas enam model data yang padat dan campuran dari para ahli (bervariasi dari $ 3.8$ hingga $ 32B parameter). Representasi adalah faktor terbesar dan kode vektor buku mengembalikan model yang dapat digunakan di mana pipa skalar PTQ runtuh. Sebagian besar yang tersisa dalam akurasi kerugian adalah pada tugas-generatif, dan jumlah model

Machine translation (Meta M2M100); the English original follows.

Abstract

Post-training quantization (PTQ) compresses the weights of a trained model with limited compute and data. Down to 4 bits it is near-lossless, but at 2 bits scalar PTQ configurations fall to chance-level accuracy. We study three choices usually treated as implementation details that matter little at 4 bits but decide 2-bit quality: the quantization representation, the calibration data, and the recovery fine-tuning data, across six dense and mixture-of experts models (ranging from $3.8$ to $32$B parameters). Representation is the largest factor and vector codebooks restore usable models where scalar PTQ pipelines collapse. Most of the remaining loss in accuracy is on \emph{generative} tasks, and the amount of performance drop is model-dependent. Furthermore, at a short recovery budget, calibration and recovery data are coupled. The best recovery data depends on the calibration data, so the two must be chosen jointly. The resulting recipe involves a vector codebook representation, a small task-aligned calibration mixture, and an inexpensive codebook-only recovery stage. Including codebooks, it uses about $2.3$ effective bits per weight and retains $89$ to $97\%$ of full-precision question-answering (QA) accuracy where scalar PTQ is at chance, and, including recovery, uses $1.4$ to $30\times$ less compute than the strongest quantization-aware-training (QAT) baseline.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Codebook, Quantization (signal processing), USable, Algorithm, Mathematics, Vector quantization

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange