Lu Sun, Xiangyi Zhang, Xiangyang Zhu, Kaiyuan Ji, Zijian Chen, Qiang Hu, Zicheng Zhang, Yuan Tian, Guangtao Zhai · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.26787 · cited by 0 works (OpenAlex) · open-access PDF
Benchmark evaluasi tradisional mengurangi pengetahuan ilmiah yang saling terkait secara alami dalam ilmu kehidupan menjadi daftar pertanyaan yang rata, mengabaikan struktur topologis dasar pengetahuan. Kami memperkenalkan, benchmark graf-struktur pertama untuk ilmu kehidupan, yang menampilkan lebih dari 10.000 pertanyaan multi-pilih berkualitas tinggi di seluruh kedokteran, biologi, dan kimia. Pendekatan kami membangun grafik evaluasi berimbang menggunakan algoritma matching dua arah dan kesamaan semantik, di mana nodes mewakili pertanyaan dan beban tepi menangkap hubungan semantik mereka. Menggunakan topologi graf ini, kami merancang dua metrik evaluasi baru. Global Coherence Score (GCS) mengukur konsistensi model di lingkungan yang terkait secara semantik, sementara Knowledge Balance Score (KBS) menganalisis model bagaimana kesalahan didistribusikan di seluruh spots konseptual grafis. LG-Bench
Machine translation (Meta M2M100); the English original follows.
Traditional evaluation benchmarks reduce inherently interconnected scientific knowledge in life sciences into flat lists of questions, disregarding the underlying topological structure of the knowledge. We introduce, the first graph-structured benchmark for life sciences, featuring over 10,000 high-quality multiple-choice questions across medicine, biology, and chemistry. Our approach constructs a weighted evaluation graph using bidirectional matching and semantic similarity algorithms, where nodes represent questions and edge weights capture their semantic relationships. Leveraging this graph topology, we design two novel evaluation metrics. The Global Coherence Score (GCS) measures a model's consistency within semantically related neighborhoods, while Knowledge Balance Score (KBS) analyzes how model errors are distributed across the graph to reveal conceptual blind spots. LG-Bench facilitates fine-grained comparison of LLMs by surfacing differences in conceptual coherence and patterns of knowledge organization across models. Our framework shifts the evaluation paradigm from flat accuracy metrics to structure-aware analysis, offering a new lens for diagnosing and improving LLM performance in the life sciences domain.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer science, Benchmark (surveying), Consistency (knowledge bases), Graph, Coherence (philosophical gambling strategy), Matching (statistics)
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26787, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange