Newest papers

Parameter-Efficient Adaptation of Large Language Models to a Low-Resource Agglutinative Language: A Corpus, Datasets, and a Reproducible Benchmark for the Bashkir Language

Mullosharaf K. Arabov, Svetlana Sergeevna Khaybullina · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31397 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

ベンチマークは、最初の2つをカバーしています。実験は、8つのアーキテクチャ(8M-8B:DistilGPT2,GPT-2ベース/メディア、Phi-3-mini-7B、Mistral-7B、YandexGPT-5-Lite-8B、MBERT、XLM-R)と3つの適応グループ(フルフィンタウンニング、QRALo=インコンテキスト学習)、継続的なプレトレーニング(CPT)を加えた(QLo on 3.8-8B classification);QLo on 3.8-3B classification; Phi-roform variability outperforms Phi-only-improforming for significant binary classification: 2.0.0.8GandexGandex-800、Mistral-500-LR=Mistral-

Machine translation (Meta M2M100); the English original follows.

Abstract

Bashkir is a low-resource agglutinative language written in Cyrillic and remains poorly represented in modern LLMs. We present an open resource suite and a systematic benchmark of adaptation methods: the Bashkir Web Corpus (71,567 documents, 46.9M tokens, 16 sources) and three labelled datasets for binary, multiclass and multilabel text classification; the benchmark covers the first two. Experiments use eight architectures (82M-8B: DistilGPT2, GPT-2 base/medium, Phi-3-mini, Mistral-7B, YandexGPT-5-Lite-8B, mBERT, XLM-R) and three adaptation groups (full fine-tuning, QLoRA, in-context learning), plus continual pretraining (CPT). (1) QLoRA on 3.8-8B models outperforms encoder-only fine-tuning on binary classification: YandexGPT-5-Lite-8B with CPT reaches macro-F1 0.748 vs 0.653 (XLM-R) and 0.650 (mBERT); pairwise differences are significant after Holm correction (p<0.001). (2) YandexGPT-5-Lite-8B significantly outperforms Mistral-7B and Phi-3-mini on multiclass classification, BPB and native-speaker ratings (mean 3.28 vs 2.52 and 2.22; n=100; alpha=0.69-0.77; p<0.05); on binary classification its QLoRA advantage is not significant (p>=0.084). (3) CPT effects depend on model and task: perplexity improves for Phi-3-mini (+0.58) and YandexGPT-5-Lite-8B (+0.33) but worsens for Mistral-7B (-1.15, p<0.001); on binary classification CPT helps Mistral-7B most (+4.0 macro-F1). (4) Tokeniser fertility on Bashkir varies 2.67x (2.92-7.80) and tracks adaptation quality, while perplexity across tokenisers is not comparable. Cyrillic tokenisers with Russian pretraining are preferable; LLM judges reproduce native ratings for grammar and coherence (rho=0.33-0.46) but not language correctness (rho=0.16, p=0.11); native evaluation remains necessary. All artefacts are open-licensed.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Perplexity, Computer science, Artificial intelligence, Agglutinative language, Benchmark (surveying), Natural language processing

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange