Newest papers

VGL-stGen: Learning a shared multimodal model for lung histology and spatial transcriptomics

Haenara Shin, Dongjoo Lee, Sungwoo Bae, Kwon Joong Na, Daeseung Lee, Hongyoon Choi · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28266 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

VGL-stGen 是从预先训练的医学视觉语言模型中采用配对肺 H&E 和空间转录与单细胞参考的多式大语言模型。培训集成图像表达和表达细胞类型对,基因条件化假标签将转录细胞类型监督转移到历史学图像中。一个部署的检查点支持十三项被评估的任务,涵盖组织和细胞类型分类,基因预测和条件表达完成,来自组合的历史学,细胞身份和观察到的基因表达。任务特定的解码生成分类标签,基因列表和结构化表达水平。为了条件完成,固定查询的基因身份,并预测它们的水平,基因分类如何自我反弹性地提高结构预测。 被控制的适应性

Machine translation (Meta M2M100); the English original follows.

Abstract

VGL-stGen is a multimodal large language model adapted from a pretrained medical vision-language model using paired lung H&E and spatial transcriptomics with single-cell references. Training integrates image-expression and expression-cell-type pairs, and gene-conditioned pseudo-labels transfer transcriptomic cell-type supervision to histology images. One deployed checkpoint supports thirteen evaluated tasks spanning tissue and cell-type classification, gene prediction and conditional expression completion from combinations of histology, cell identity and observed gene expression. Task-specific decoding produces categorical labels, gene lists and structured expression levels. For conditional completion, fixing queried gene identities and predicting their levels autoregressively improves structured prediction. Controlled adaptation shows that restricting supervision to requested expression-level tokens can further improve image-conditioned completion. Image-permutation experiments indicate that correctly paired histology adds information beyond observed expression. External classification relies partly on computational references with shared annotation lineage, and some H&E sources were exposed during cancer training or model development. Within these conditions, VGL-stGen demonstrates how a medical vision-language backbone can learn from spatial transcriptomics and histology to connect tissue morphology and molecular states through shared generative modeling.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Transcriptome, Computer science, Artificial intelligence, Categorical variable, Annotation, Adaptation (eye)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange