Newest papers

VERA: A Responsible AI Dashboard for Assessing LLM Deployments in Light of the EU AI Act

Robin Quériaux, Solveig Bachellery, Léa Déléris, Saurabhsingh Rajput, Mariam Barry, Tushar Sharma, Jérôme Lebecq · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31588 · cited by 0 works (OpenAlex) · open-access PDF

Korean

규제된 환경에서 대형 언어 모델을 배포하는 기관은 배포를 승인하기 전에 책임있는 AI 관행에 대한 기술적 증거가 종종 필요합니다. COMPL-AI는 EU AI Act에 대한 기술적 해석을 제공하며이 증거를 6 개 원칙 그룹과 18 개의 기술적 요구 사항, 12 개의 현재 벤치마크를 측정 할 수 있습니다.이 증거를 생산하는 벤치마크 스위트는 일반적으로 명령 라인에서 운영되며 배포를 승인하는 직원은 공공 엔지니어가 아닙니다. 우리는 대규모 유럽 은행 내에서 책임있는 AI 프로세스를 촉진하기 위해 배포 된 오픈 소스 자원 보드 인 VERA를 제시합니다. 리뷰는 모델을 선택하고 공개 벤치마크를 실행합니다. VERA는 현재

Machine translation (Meta M2M100); the English original follows.

Abstract

Institutions that deploy large language models in regulated settings often need technical evidence of responsible AI practices before approving a deployment. COMPL-AI provides a technical interpretation of the EU AI Act that organises this evidence under six principle groups and eighteen technical requirements, twelve of which current benchmarks can measure. The benchmark suites that produce this evidence are typically operated from a command line, while the staff who approve deployments are non-engineers. We present VERA, an open-source self-hosted dashboard deployed to facilitate responsible AI processes within a large European bank. A reviewer selects a model and runs public benchmarks against it. VERA scores the twelve requirements that current benchmarks can measure and instruments the energy consumed by each evaluation, adding run-level evidence for environmental impact. We run four served open-weight models through the requirement pipeline, and the dashboard displays a score for each measurable requirement. A within-subject study then compares how participants read these benchmark results: from the raw output files in one condition, and through the VERA dashboard in the other. The dashboard reduced the time needed to perform the responsible AI assessment, reducing the time required from 56 seconds using raw files to 30 seconds using our dashboard, on average. Our dashboard scored highly (4.36 out of 5) on usefulness in the survey conducted within the organization. The dashboard and the study material are released publicly to promote replicability and reproducibility in different organisational contexts.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Dashboard, Benchmark (surveying), Raw data, Computer science, Benchmarking, Measure (data warehouse)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange