The AI Barometer NeoMundi is our weekly and longitudinal observatory dedicated to the real-world monitoring of generative AI systems behaviors.
Each week, we analyze a panel of 12 anonymized model profiles through 4 fixed questions repeated 100 times each (4,800 executions in total). We measure and score several key dimensions: observable stability, factual validity, information density, factual risk signals, semantic variation and cost efficiency.
These continuous observations help detect behavioral trends, subtle evolutions and silent regime shifts that traditional point-in-time benchmarks cannot capture.
Methodology and public data
This barometer is based on an open, transparent and reproducible methodology. This cartography is generated from aggregated public data by a Python script. The results rely on the NeoMundi public baseline and on the Observatory’s methodology.
The Python scripts, scoring protocols, raw datasets and aggregated results are published on GitHub. The scores are measurement signals, never verdicts or rankings.
Find here all editions of the Artificial Intelligence Barometers, with their detailed analyses and specific observations.
All AI Barometers When an AI responds consistently… but no longer says exactly the same thing This week, the Barometer highlights a particularly subtle form of instability: for the same question, the same system can generate multiple plausible, well-written, and seemingly logical responses… while varying the conclusion, nuance, or level of caution. None of the […]
All AI Barometers When a response sounds perfectly solid… but still warrants verification This week, the Barometer highlights a particularly insidious phenomenon: responses formulated with confidence, plausible at first glance, yet containing precise factual errors. Answers that many readers might accept without question. This is exactly what our measurement instrument detected on multiple occasions. Three
All AI Barometers One profile shows a clear shift in zone, while the overall picture remains calm This week, the overall average remains calm. However, one of the observed systems has changed its behavior and triggered our response verification control much more frequently compared to Barometer #1 last week. Indeed, the anonymized LLM with the
All AI Barometers Building a baseline To produce this first Barometer, we first established a public baseline. It was built from 3 execution waves × 12 de-identified systems × 4 fixed questions × 100 repetitions, representing 14,400 completed observations. This baseline serves as the reference point for measuring changes observed in future weekly publications. View
