Lu Sun, Xiangyi Zhang, Xiangyang Zhu, Kaiyuan Ji, Zijian Chen, Qiang Hu, Zicheng Zhang, Yuan Tian, Guangtao Zhai · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.26787 · cited by 0 works (OpenAlex) · open-access PDF
传统的评估基准将生命科学中固有相互关联的科学知识减少到平板列表,忽略了知识的底层 topological 结构。 我们引入了生命科学的第一个图形结构基准,在医学,生物学和化学中提供了超过 10,000 个高质量的多选项问题。我们的方法使用双向匹配和语义相似性算法构建一个重量级评估图表,在那里节点代表问题,边缘重量捕捉到他们的语义关系。利用此图形 topology,我们设计了两种新的评估指标。 全球一致性得分(GCS)衡量模型的一致性在语义相关的邻里,而知识平衡得分(KBS)分析了错误如何分布在图形揭示概念的盲点上。 我们的LG-Bench
Machine translation (Meta M2M100); the English original follows.
Traditional evaluation benchmarks reduce inherently interconnected scientific knowledge in life sciences into flat lists of questions, disregarding the underlying topological structure of the knowledge. We introduce, the first graph-structured benchmark for life sciences, featuring over 10,000 high-quality multiple-choice questions across medicine, biology, and chemistry. Our approach constructs a weighted evaluation graph using bidirectional matching and semantic similarity algorithms, where nodes represent questions and edge weights capture their semantic relationships. Leveraging this graph topology, we design two novel evaluation metrics. The Global Coherence Score (GCS) measures a model's consistency within semantically related neighborhoods, while Knowledge Balance Score (KBS) analyzes how model errors are distributed across the graph to reveal conceptual blind spots. LG-Bench facilitates fine-grained comparison of LLMs by surfacing differences in conceptual coherence and patterns of knowledge organization across models. Our framework shifts the evaluation paradigm from flat accuracy metrics to structure-aware analysis, offering a new lens for diagnosing and improving LLM performance in the life sciences domain.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer science, Benchmark (surveying), Consistency (knowledge bases), Graph, Coherence (philosophical gambling strategy), Matching (statistics)
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26787, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange