As artificial intelligence systems become increasingly multimodal and capable of reasoning across text, images and other data modalities, ensuring that these models are trustworthy requires more than stronger architectures alone. Robust evaluation frameworks and meaningful benchmarks are becoming essential tools for understanding model capabilities, limitations and potential risks.
This challenge was at the centre of AIXPERT partner the Vector Institute’s contribution to the Helmholtz AI Conference 2026: AI for Science (HAICon 2026), held in Munich from 8–11 June 2026. The conference brought together leading AI researchers, consultants and industry representatives to discuss the latest advances in AI for scientific discovery, with a strong emphasis on trustworthy, transparent and responsible AI.
During the conference’s Workshop and Tutorial Day, Dr. Shaina Raza (Vector Institute) presented HumaniBench: A Human-Centric Benchmark for Large Multimodal Models Evaluation as part of the workshop “Current Status of the Benchmarking Field: Lessons Learned from the First Half of the UNLOCK Initiative.” The session explored emerging best practices for designing AI benchmarks that can support reliable evaluation across rapidly evolving research domains.
Putting humans at the centre of multimodal AI evaluation
Large Multimodal Language Models (MLLMs) are rapidly expanding into domains where decisions have real-world consequences, from healthcare and scientific research to education and industrial applications. While their capabilities continue to improve, evaluating these systems remains a significant challenge.
Dr. Raza’s presentation introduced HumaniBench, a human-centric approach to evaluating multimodal AI systems that moves beyond traditional performance metrics. By focusing on evaluation methodologies that better reflect human expectations, reasoning and reliability, the work contributes to a growing international effort to develop more meaningful assessments of trustworthy AI.
The workshop highlighted that effective benchmarking is no longer simply about comparing model accuracy. It also involves understanding robustness, safety, transparency and reproducibility—qualities that are increasingly important as foundation models are deployed in scientific and high-impact environments.
Advancing trustworthy AI through benchmarking
The session brought together experts working on benchmarking across diverse scientific domains, including chemistry, single-cell genomics, AI energy consumption and AI safety. The programme concluded with a panel discussion examining how benchmark design influences both scientific progress and responsible AI development.
For AIXPERT, these discussions closely align with the project’s mission to develop trustworthy, explainable and human-centred AI technologies. Reliable evaluation frameworks are a critical foundation for ensuring that advanced AI systems can be deployed with confidence across complex real-world environments.
By contributing expertise in multimodal AI evaluation, the Vector Institute helps strengthen the broader research community’s understanding of how trustworthy AI can be assessed, compared and continuously improved. As benchmarking practices continue to evolve, initiatives such as HumaniBench will play an increasingly important role in building transparent and dependable AI systems for science and society.
