If an AI system changes its behavior without changing its name, how would we notice? A single response is not enough. The same system can produce different answers to an identical question. Its behavior can also evolve over time, particularly when its model, settings, or infrastructure change. AI Weather™ makes some of these variations visible. […]
All AI Barometers Observation level: 🟠 Orange — insufficient coverage Of 4,800 executions launched across 12 systems, 3,981 were fully measured, representing 82.94% coverage, below the 90% standard publication threshold. Mean stability stands at 0.922, with an overall semantic variation rate of 1.61%. Two systems produced no usable observations despite multiple attempts, accounting for 800
The NeoMundi team is pleased to take part in an educational initiative designed to help students better understand AI Weather and the reliability issues associated with generative artificial intelligence systems. Follow the #AIweather on X This initiative is exclusively educational. Its purpose is to help students become better-informed users of artificial intelligence: to understand that
All AI Barometers Observation level: 🟡 Yellow Across 4,800 executions distributed among 12 systems, AI Barometer #12 remains overwhelmingly within the normal regime. Average stability remains virtually unchanged at approximately 0.923. The overall semantic variation rate stands at 1.41%, compared with 1.93% in Barometer #11. Coverage, however, decreased from 99.52% to 95.90%, with 197 incomplete
All AI Barometers Observation Level: 🟢 Green Across 4,800 executions distributed among 12 systems, AI Barometer #11 remains largely in the normal regime, with average stability virtually unchanged around 0.923. Global semantic variation reaches 1.93%, compared with 1.57% in Barometer #10. Factual alerts remain limited, and no generalized shift of the panel is observed. Reading:
All AI Barometers Observation Level: 🟢 Green Across 4,800 executions distributed among 12 systems, AI Barometer #10 remains largely in the normal regime, with average stability virtually unchanged at 0.9229. Global semantic variation decreases to 1.57%, compared with 2.09% in Barometer #9, while factual alerts remain limited. The observed movement remains mainly localized on a
Two systems with almost the same stability can differ by more than 30 points in factuality Protocol 1 – 12 systems × 790 TruthfulQA questions NeoMundi’s August monthly cartography once again compares twelve de-identified AI profiles across several complementary dimensions: their behavioral stability over 790 TruthfulQA questions, the factuality of their responses independently assessed by
All AI Barometers Observation Level: 🟢 Green Across 4,800 executions distributed among 12 systems, Barometer #9 remains largely in the normal regime (97.12%), with average stability virtually unchanged at 0.9228. Semantic variation, however, rises to 2.06%, compared with approximately 1.06% in the 8th Barometer, while factual alerts remain limited. The observed movement is mainly localized
All AI Barometers Observation Level: 🟢 Green This week, AI Barometer #8 confirms a return to global calm and moves to the green level. Two movements are particularly notable: This edition also presents a new example of a fragile factual claim embedded in a fluent and convincing response. This case illustrates the potential value of
All AI Barometers Observation Level: 🟡 Yellow This week, AI Barometer #7 highlights two complementary dimensions. On the one hand, the overall calm observed in AI Barometer #6 is confirmed: average semantic variation decreases, and the majority of profiles remain close to their usual regime. On the other hand, this return to calm is not
