⚠️ The first version of this Artificial Intelligence cartography combined two distinct protocols: a large-scale campaign conducted on twelve profiles (LLMs) submitted to 790 questions from TruthfulQA, and a complementary campaign based on twelve profiles, three categories of questions and 150 observations per category.This combination made it possible to observe multiple dimensions of system behavior […]
Companies no longer need to be told that artificial intelligence carries risks. They already know. They also know that a system can produce a convincing yet fragile answer, that a provider can modify a model or its routing without the change being immediately visible, and that an application can remain fully available from a technical
All AI Barometers When an AI responds consistently… but no longer says exactly the same thing This week, the Barometer highlights a particularly subtle form of instability: for the same question, the same system can generate multiple plausible, well-written, and seemingly logical responses… while varying the conclusion, nuance, or level of caution. None of the
All AI Barometers When a response sounds perfectly solid… but still warrants verification This week, the Barometer highlights a particularly insidious phenomenon: responses formulated with confidence, plausible at first glance, yet containing precise factual errors. Answers that many readers might accept without question. This is exactly what our measurement instrument detected on multiple occasions. Three
All AI Barometers One profile shows a clear shift in zone, while the overall picture remains calm This week, the overall average remains calm. However, one of the observed systems has changed its behavior and triggered our response verification control much more frequently compared to Barometer #1 last week. Indeed, the anonymized LLM with the
All AI Barometers Building a baseline To produce this first Barometer, we first established a public baseline. It was built from 3 execution waves × 12 de-identified systems × 4 fixed questions × 100 repetitions, representing 14,400 completed observations. This baseline serves as the reference point for measuring changes observed in future weekly publications. View
1. For AI systems, price, stability and factual accuracy do not always move together We measured 12 major generative AI systems using the same protocol, in order to compare their observable stability, factual validity and relative cost per request. Each point represents an observed profile. Horizontal axis: observable stability · Vertical axis: evaluated factual validity
1. MURMURATION – Research on Collective Runtime Stability Distributed autonomous systems can now coordinate trajectories, distribute tasks, and maintain dynamic formations at scale. But one critical question remains largely unsolved: how can we measure the collective stability of a multi-agent system before a systemic failure in cohesion emerges? Current distributed architectures are designed to optimize:
