Newest papers

GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation

Quang Anh Kiet Tran, Nam Anh Tran, Manh Hung Trinh, Thanh Danh Le, Khang Nhat Hoang Vo, Huynh Dai Nhan Tran · 2026 · arXiv

DOI: 10.48550/arXiv.2610.30067 · open-access PDF

Japanese

ベトナムの事前訓練されたエンコーダーは急速に改善したが、最も強力なものは、大規模で多言語の語彙モデルであるか、または外部の単語セグメントに依存しているため、導入するのに費用がかかる。我々は、26Mと49Mのパラメータで直接原文で動作する2つのコンパクトベトナムエンコーダーのファミリーであるGoiBERTを紹介する。

Machine translation (Meta M2M100); the English original follows.

Abstract

Vietnamese pre-trained encoders have improved rapidly, but the strongest ones are large, multilingual-vocabulary models or depend on external word segmentation, which makes them costly to deploy. We present GoiBERT, a family of two compact Vietnamese encoders with 26M and 49M parameters that operate directly on raw text. GoiBERT learns from two complementary teachers. A same-tokenizer Vietnamese encoder first teaches general-purpose representations. A much larger teacher then provides task-level guidance through soft labels that are generated without ever labeling its own training data. Across

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange