Newest papers

VGL-stGen: Learning a shared multimodal model for lung histology and spatial transcriptomics

Haenara Shin, Dongjoo Lee, Sungwoo Bae, Kwon Joong Na, Daeseung Lee, Hongyoon Choi · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28266 · cited by 0 works (OpenAlex) · open-access PDF

Korean

VGL-stGen은 한 세포 참조와 함께 쌍화 된 폐 H&E 및 공간 transcriptomics를 사용하여 사전 훈련 된 의학 시각 언어 모델에서 적응 한 대형 언어 모델입니다. 교육은 이미지 표현 및 표현 세포 유형 쌍을 통합하고 유전자 조건화 된 pseudo-etiquettes는 세포 유형 감독을 histology 이미지로 전송합니다. 하나의 배포 체크 포인트는 조직과 세포 유형 분류, 유전자 예측 및 조건형 표현 완료를 포함하는 13 개의 평가 작업을 지원합니다. 임무 특정 디코딩은 분류 라벨, 유전자 목록 및 구조화 된 표현 수준을 생성합니다. 조건적 완료를 위해, 쿼리 된 유전자 정체를 고정하고 그들의 수준을 예측하는 것은 자극적으로 구조화

Machine translation (Meta M2M100); the English original follows.

Abstract

VGL-stGen is a multimodal large language model adapted from a pretrained medical vision-language model using paired lung H&E and spatial transcriptomics with single-cell references. Training integrates image-expression and expression-cell-type pairs, and gene-conditioned pseudo-labels transfer transcriptomic cell-type supervision to histology images. One deployed checkpoint supports thirteen evaluated tasks spanning tissue and cell-type classification, gene prediction and conditional expression completion from combinations of histology, cell identity and observed gene expression. Task-specific decoding produces categorical labels, gene lists and structured expression levels. For conditional completion, fixing queried gene identities and predicting their levels autoregressively improves structured prediction. Controlled adaptation shows that restricting supervision to requested expression-level tokens can further improve image-conditioned completion. Image-permutation experiments indicate that correctly paired histology adds information beyond observed expression. External classification relies partly on computational references with shared annotation lineage, and some H&E sources were exposed during cancer training or model development. Within these conditions, VGL-stGen demonstrates how a medical vision-language backbone can learn from spatial transcriptomics and histology to connect tissue morphology and molecular states through shared generative modeling.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Transcriptome, Computer science, Artificial intelligence, Categorical variable, Annotation, Adaptation (eye)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange