Newest papers

VERA: A Responsible AI Dashboard for Assessing LLM Deployments in Light of the EU AI Act

Robin Quériaux, Solveig Bachellery, Léa Déléris, Saurabhsingh Rajput, Mariam Barry, Tushar Sharma, Jérôme Lebecq · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31588 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

部署大型语言模型在受管制的环境中的机构往往需要在批准部署之前对负责任的AI实践的技术证据。COMPL-AI提供欧盟AI法案的技术解释,该法案在六个原则组和十八项技术要求下组织这种证据,其中十二个可以测量当前的基准。生产这种证据的基准套件通常由命令线操作,而批准部署的员工是非公共工程师。我们介绍VERA,一个开放源代码的自我托管仪表板,用于促进欧洲大型银行内的负责任的AI流程。评审员选择一个模型,并对其运行公共基准套件。VERA研究了当前基准可以测量的十二项要求,并为每个评估所消耗的能量提供了工具,增加了对环境影响的原始级别的证据。

Machine translation (Meta M2M100); the English original follows.

Abstract

Institutions that deploy large language models in regulated settings often need technical evidence of responsible AI practices before approving a deployment. COMPL-AI provides a technical interpretation of the EU AI Act that organises this evidence under six principle groups and eighteen technical requirements, twelve of which current benchmarks can measure. The benchmark suites that produce this evidence are typically operated from a command line, while the staff who approve deployments are non-engineers. We present VERA, an open-source self-hosted dashboard deployed to facilitate responsible AI processes within a large European bank. A reviewer selects a model and runs public benchmarks against it. VERA scores the twelve requirements that current benchmarks can measure and instruments the energy consumed by each evaluation, adding run-level evidence for environmental impact. We run four served open-weight models through the requirement pipeline, and the dashboard displays a score for each measurable requirement. A within-subject study then compares how participants read these benchmark results: from the raw output files in one condition, and through the VERA dashboard in the other. The dashboard reduced the time needed to perform the responsible AI assessment, reducing the time required from 56 seconds using raw files to 30 seconds using our dashboard, on average. Our dashboard scored highly (4.36 out of 5) on usefulness in the survey conducted within the organization. The dashboard and the study material are released publicly to promote replicability and reproducibility in different organisational contexts.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Dashboard, Benchmark (surveying), Raw data, Computer science, Benchmarking, Measure (data warehouse)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange