The NeoMundi AI Observatory is our public research program dedicated to the continuous, transparent and reproducible evaluation of generative AI systems.
We regularly publish thermodynamic cartographies and in-depth observations, including the NeoMundi AI Barometer, which tracks de-identified model profiles over time across multiple dimensions: stability, factual validity, informational density, factual-risk signals, semantic variation and cost efficiency. These publications reveal behavioral trends and silent regime changes that traditional one-shot benchmarks cannot capture.
All our work is built on open methodologies and publicly available data. Generation scripts, scoring protocols and raw datasets are released on GitHub, allowing researchers, developers and the wider AI community to audit, reproduce and build upon our observations.
⚠️ The first version of this Artificial Intelligence cartography combined two distinct protocols: a large-scale campaign conducted on twelve profiles (LLMs) submitted to 790 questions from TruthfulQA, and a complementary campaign based on twelve profiles, three categories of questions and 150 observations per category.This combination made it possible to observe multiple dimensions of system behavior […]
Companies no longer need to be told that artificial intelligence carries risks. They already know. They also know that a system can produce a convincing yet fragile answer, that a provider can modify a model or its routing without the change being immediately visible, and that an application can remain fully available from a technical
All AI Barometers When an AI responds consistently… but no longer says exactly the same thing This week, the Barometer highlights a particularly subtle form of instability: for the same question, the same system can generate multiple plausible, well-written, and seemingly logical responses… while varying the conclusion, nuance, or level of caution. None of the
By Frédéric Dumollard, former field inspector, then general agent and insurance broker. After years spent at the heart of underwriting corporate liability risks, he is now co-founder of NeoMundi and brings his expert perspective on the new insurance challenges posed by generative AI. DOI: 10.5281/zenodo.21222308 Why a former underwriter is taking up this subject I
All AI Barometers When a response sounds perfectly solid… but still warrants verification This week, the Barometer highlights a particularly insidious phenomenon: responses formulated with confidence, plausible at first glance, yet containing precise factual errors. Answers that many readers might accept without question. This is exactly what our measurement instrument detected on multiple occasions. Three
NeoMundi is pleased to announce its acceptance into the NVIDIA Inception program. An Already Operational Runtime Governance Infrastructure NeoMundi joins NVIDIA Inception to accelerate the development of ControlTower, our infrastructure for measurement, auditability, and real-time governance of AI systems. As organizations massively deploy AI in production, the key question is no longer whether a model
All AI Barometers One profile shows a clear shift in zone, while the overall picture remains calm This week, the overall average remains calm. However, one of the observed systems has changed its behavior and triggered our response verification control much more frequently compared to Barometer #1 last week. Indeed, the anonymized LLM with the
In the framework of the NeoMundi AI Observatory’s work, we are publishing today a contribution by James Moore, founder of Nova Jema AI Systems and specialist in the governance of artificial intelligence systems under real-world execution conditions. He kindly shared with us a precise reflection on a frequently overlooked subject: once we measure an AI’s
All AI Barometers Building a baseline To produce this first Barometer, we first established a public baseline. It was built from 3 execution waves × 12 de-identified systems × 4 fixed questions × 100 repetitions, representing 14,400 completed observations. This baseline serves as the reference point for measuring changes observed in future weekly publications. View
Scientific Contributor / Data Scientist – NeoMundi AI Observatory Most discussions on AI governance start too late. We believe measurement must come first. We have developed ControlTower, a diagnostic, piloting, and traceability instrument designed to continuously observe the risk of AI responses in real-world conditions. ControlTower enables standardized and reproducible measurement campaigns, allows comparison of
