The problem with plausible answers: AIXPERT findings at EMNLP 2026 

Artificial Intelligence • GenAI • Explainable AI • Multi-Agent Systems • Explainable Multimodal Large Language Models • Context-Aware Systems •  

From misleading industrial AI recommendations to hidden biases in multimodal models, AIXPERT researchers presented new findings at EMNLP 2026 in Budapest, advancing the scientific foundations of more reliable, explainable and trustworthy AI. 

EMNLP 2026 brought the international natural language processing research community to Budapest, Hungary, from 24–29 October. 

Artificial intelligence can produce remarkably accurate answers while still making mistakes that are difficult to detect. An industrial AI assistant might recommend a troubleshooting step that exists in the official manual but is entirely inappropriate for the situation. A language model might solve complex mathematical problems yet fail to acknowledge when a question has no solution. 

Understanding these limitations is becoming increasingly important as AI enters professional and high-stakes environments. At the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP), held in Budapest from 24–29 October, AIXPERT partners Athena Research Center’s Institute for Language and Speech Processing (ILSP), the University of Groningen and the Vector Institute contributed five research papers exploring how AI systems can be better understood, evaluated and improved. 

Scroll to the bottom of the page to find more resources on each paper.

Evaluating How Well AI Understands Real-World Audio and Video 

Ahmed Y. Radwan (Vector Institute) presented SONIC-O1 virtually at EMNLP 2026.

How effectively can multimodal AI interpret real-world interactions involving speech, video and human behaviour? Researchers from the Vector Institute and University of Groningen addressed this question in SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding (A. Y. Radwan et al., 2026). Their benchmark evaluates AI models using approximately 60 hours of audiovisual material across 13 conversational settings, from medical consultations to job interviews. The findings revealed difficulties in identifying when events occur, alongside performance disparities of up to 21.4% across perceived demographic groups in temporal localization tasks. Such results highlight the need for more comprehensive and fairness-aware AI evaluations.

Evaluating Robustness of LLM-based Assistants Against Real Industrial Language

Another study examined the robustness of AI-assisted troubleshooting in industrial maintenance. In DiagFlowBench: Evaluating How Language Models Handle Off-Procedure Inputs in Grounded Diagnostic Dialogue (G. Gil de Avalle et al., 2026), researchers from the University of Groningen and Vector Institute evaluated the abstention capabilities of language models when operator prompt unexpected inputs, using a benchmark they authored composed of 50 maintenance flowcharts from a consumer manufacturers and 1676 conversations between a maintenance operator and an AI assistant. 

Guillermo Gil de Avalle (University of Groningen) presented DiagFlowBench at EMNLP 2026. 

Their findings exposed a subtle but consequential hallucination type: models frequently recommended valid procedural steps from the flowchart that were inappropriate for the situation; a phenomenon termed forced mapping. For manufacturers and industrial AI developers, the research underscores the importance of training unexpected behavior into the systems, building robustness, and preventing even the inconspicuous failures.  

Understanding How Language Models Reason and Process Information

Nikolaos Xiros presenting the CoRM research at EMNLP 2026

Two contributions from Athena Research Center’s Institute for Language and Speech Processing (ILSP) explored how language models organise information internally and how faithfully their responses reflect what they have learned. In Beyond Magnitude: Contrastive Routing for Modular Mixture-of-Experts (N. Xiros et al., 2026), researchers introduced the Contrastive Routing Mechanism (CoRM), a new approach that helps specialised components within language models process information more effectively. The method improved performance across nine reasoning and language understanding benchmarks with relatively modest computational overhead, offering new possibilities for more efficient and specialised AI architectures. 

Meanwhile, More Capable, Less Faithful: A Multilingual Analysis of Mathematical (Un)Solvability Detection in LLMs (M.-E. Zoumpoulidi et al., 2026), presented at the associated Multilingual Representation Learning workshop (MRL 2026), investigated mathematical reasoning in English, French and Greek. 

The researchers found that stronger mathematical reasoning does not necessarily translate into more faithful responses. Several models performed better in English while showing greater consistency between internal solvability assessments and generated responses in French or Greek. The findings have implications for developing reliable multilingual AI systems. 
 

The research poster for “More Capable, Less Faithful” at the MRL 2026 workshop, held alongside EMNLP in Budapest.  

Using AI to better understand scientific knowledge 

The fifth contribution from Athena Research Center’s ILSP explored how AI could help researchers navigate the rapidly expanding scientific literature. In Towards Multidisciplinary Citance Analysis: A Novel Approach for Intent, Polarity, and Semantics Classification (G. Pantelakis et al., 2026), researchers introduced CitAn, a framework and dataset covering 1,165 citation instances across 80 scientific subfields. Rather than simply counting citations, CitAn examines why a study is referenced, what aspect of it is being cited and whether the citing author supports or challenges it. 

The approach offers opportunities for more sophisticated literature analysis, knowledge discovery and research impact assessment, with potential applications for researchers, scientific publishers and funding organisations. 

Towards more trustworthy AI 

Together, these five contributions reflect a central ambition of AIXPERT: moving beyond what AI systems can achieve to understanding how they behave, where they fail and how their reliability can be improved. 

From real-world benchmarking and industrial decision support to model interpretability and scientific knowledge discovery, AIXPERT researchers are helping advance the methods needed to make increasingly capable AI systems more transparent, accountable and trustworthy. 

Radwan, A. Y., Emmanouilidis, C., Tabassum, H., Pandya, D., & Raza, S. (2026). SONIC-O1: A real-world benchmark for evaluating multimodal large language models on audio-video understanding. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (pp. 17786–17825). Association for Computational Linguistics. https://aclanthology.org/2026.emnlp-main.767/

Research paper | Presentation | Benchmark website

Gil de Avalle, G., Maruster, L., Raza, S., & Emmanouilidis, C. (2026). DiagFlowBench: Evaluating how language models handle off-procedure inputs in grounded diagnostic dialogue. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track (pp. 1854–1871). Association for Computational Linguistics. https://aclanthology.org/2026.emnlp-industry.108/

Research paper | Research poster

Xiros, N., Damianos, D., Zoumpoulidi, M.-E., Voukoutis, L., Katsouros, V., & Paraskevopoulos, G. (2026). Beyond magnitude: Contrastive routing for modular mixture-of-experts. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (pp. 17267–17280). Association for Computational Linguistics. https://aclanthology.org/2026.emnlp-main.745/

Research paper | Open-source code

 

Zoumpoulidi, M.-E., Xiros, N., & Paraskevopoulos, G. (2026). More capable, less faithful: A multilingual analysis of mathematical (un)solvability detection in LLMs. In Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026) (pp. 58–67). Association for Computational Linguistics. https://aclanthology.org/2026.mrl-1.6/

Research paper | Open-source code

Pantelakis, G., Stavropoulos, P., & Papageorgiou, H. (2026). Towards multidisciplinary citance analysis: A novel approach for intent, polarity, and semantics classification. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (pp. 60882–60907). Association for Computational Linguistics. https://aclanthology.org/2026.emnlp-main.2677/

Research paper