Newest papers

VGL-stGen: Learning a shared multimodal model for lung histology and spatial transcriptomics

Haenara Shin, Dongjoo Lee, Sungwoo Bae, Kwon Joong Na, Daeseung Lee, Hongyoon Choi · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28266 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

VGL-stGen adalah model bahasa besar multimodal yang disesuaikan dari model morfologi visual medis pra-latih menggunakan H&E paru-paru dan transcriptomics spasial dengan referensi sel tunggal. Pelatihan mengintegrasikan pasangan ekspresi gambar dan ekspresi-sel-type, dan gen-conditioned pseudo-label mentransfer pengawasan transcriptomic tipe sel ke gambar histologi. Satu titik kontrol yang didistribusikan mendukung tiga belas tugas yang dinilai yang mencakup jaringan dan klasifikasi jenis sel, prediksi gen dan ekspresi kondisional dari kombinasi histologi, identitas sel dan ekspresi gen yang diamati. Dekoding spesifik tugas menghasilkan label kategori, daftar gen dan tingkat ekspresi terstruktur. Untuk penyelesaian kondisional, memperbaiki identitas gen yang diminta dan memprediksi tingkat mereka secara otomatis meningkatkan prediksi ter

Machine translation (Meta M2M100); the English original follows.

Abstract

VGL-stGen is a multimodal large language model adapted from a pretrained medical vision-language model using paired lung H&E and spatial transcriptomics with single-cell references. Training integrates image-expression and expression-cell-type pairs, and gene-conditioned pseudo-labels transfer transcriptomic cell-type supervision to histology images. One deployed checkpoint supports thirteen evaluated tasks spanning tissue and cell-type classification, gene prediction and conditional expression completion from combinations of histology, cell identity and observed gene expression. Task-specific decoding produces categorical labels, gene lists and structured expression levels. For conditional completion, fixing queried gene identities and predicting their levels autoregressively improves structured prediction. Controlled adaptation shows that restricting supervision to requested expression-level tokens can further improve image-conditioned completion. Image-permutation experiments indicate that correctly paired histology adds information beyond observed expression. External classification relies partly on computational references with shared annotation lineage, and some H&E sources were exposed during cancer training or model development. Within these conditions, VGL-stGen demonstrates how a medical vision-language backbone can learn from spatial transcriptomics and histology to connect tissue morphology and molecular states through shared generative modeling.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Transcriptome, Computer science, Artificial intelligence, Categorical variable, Annotation, Adaptation (eye)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange