Lu Sun, Xiangyi Zhang, Xiangyang Zhu, Kaiyuan Ji, Zijian Chen, Qiang Hu, Zicheng Zhang, Yuan Tian, Guangtao Zhai · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.26787 · cited by 0 works (OpenAlex) · open-access PDF
전통적인 평가 벤치마크는 생명 과학에서 본질적으로 상호 연결된 과학적 지식을 질문의 평평한 목록으로 줄이고, 지식의 근본적인 토폴로그 구조를 무시합니다. 우리는 인생 과학에 대한 최초의 그래프 구조적 벤치마크를 소개하며, 의학, 생물학 및 화학 전반에 걸쳐 10,000 개가 넘는 고품질의 다중 선택 질문을 특징으로합니다. 우리의 접근 방식은 양방향 일치 및 상징적 유사성 알고리즘을 사용하여 가중계 평가 그래프를 구축하며, 노드가 질문을 나타내고 가장자리 무게가 그들의 상징적 관계를 캡처합니다.이 그래프 토폴로그를 활용함으로써 우리는 두 가지 새로운 평가 측정을 설계합니다. 글로벌 일관성 점수 (GCS)
Machine translation (Meta M2M100); the English original follows.
Traditional evaluation benchmarks reduce inherently interconnected scientific knowledge in life sciences into flat lists of questions, disregarding the underlying topological structure of the knowledge. We introduce, the first graph-structured benchmark for life sciences, featuring over 10,000 high-quality multiple-choice questions across medicine, biology, and chemistry. Our approach constructs a weighted evaluation graph using bidirectional matching and semantic similarity algorithms, where nodes represent questions and edge weights capture their semantic relationships. Leveraging this graph topology, we design two novel evaluation metrics. The Global Coherence Score (GCS) measures a model's consistency within semantically related neighborhoods, while Knowledge Balance Score (KBS) analyzes how model errors are distributed across the graph to reveal conceptual blind spots. LG-Bench facilitates fine-grained comparison of LLMs by surfacing differences in conceptual coherence and patterns of knowledge organization across models. Our framework shifts the evaluation paradigm from flat accuracy metrics to structure-aware analysis, offering a new lens for diagnosing and improving LLM performance in the life sciences domain.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer science, Benchmark (surveying), Consistency (knowledge bases), Graph, Coherence (philosophical gambling strategy), Matching (statistics)
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26787, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange