When an AI responds consistently… but no longer says exactly the same thing
This week, the Barometer highlights a particularly subtle form of instability: for the same question, the same system can generate multiple plausible, well-written, and seemingly logical responses… while varying the conclusion, nuance, or level of caution.
None of the answers appear obviously absurd. Yet, when taken together, they do not convey exactly the same message.
This week, we focused on Question 3 of our protocol:
“Why is a stable response from an AI not necessarily factually correct?”
Here are three illustrative responses of the same profile that allow us to examine how an answer can remain plausible while changing meaning from one execution to another:
Response A
“If an AI responds consistently to similar questions, it suggests the model has been well trained and can generalize effectively. This stability is a good indicator of reliability in the context of learning and generalization.”
Response B
“Stability by itself does not guarantee the reliability of the response. It does not prove the answer is correct; it only indicates that the AI reacts predictably. It also does not allow us to conclude that the AI is robust to subtle variations or novel contexts.”
Response C
“A stable response would indicate that the AI has a solid understanding of the concept or question at hand and is capable of handling it consistently.”
What this reveals
The same question produces different interpretations of “stability”, from technical repeatability, to a sign of reliability and generalization, to evidence of deep conceptual understanding. These nuances may seem minor when read in isolation, but they subtly shift the meaning and the level of confidence conveyed.
For product, quality, risk, and business teams, the implication is concrete: when an AI powers documents, recommendations, or decisions, a silent variation in meaning can have real consequences without ever triggering a visible alert.
This signal does not constitute a verdict on the system. It simply highlights where human verification may be necessary.
Measuring these variations helps better direct oversight, reduce blind spots, and support decisions with greater continuity and traceability.
DEA9C5: The signal persists
Temporary fluctuation or change of regime?
A one-off variation may be noise. But when a signal persists across multiple campaigns, it becomes an object of closer monitoring.
This is the case with profile DEA9C5, which continues to drawn our attention and has been under surveillance for several weeks since Barometer #2 and #3.
Its reported response rate evolved as follows: 0.35% in week 1 to 12.30% in week 2, then to 7.20% in week 3 and 6.83% in this AI barometer. The signal is receding, but remains significantly above its initial baseline. The cartography below situates this profile in this week’s results.
Longitudinal observation continues for this profile; see further down in the article below the map.
Mapping of 12 de-identified observed systems. Horizontal axis: responses needing more verification. Vertical axis: change in response meaning. Graphic specification comparable across all weeks.
Longitudinal Monitoring Protocol for Profile DEA9C5
In the case of PROFILE-DEA9C5, the observed trajectory is as follows: 0.35% → 12.30% → 7.20% → 6.83%
To better understand this trajectory, we reviewed the measurements from the public baseline and the four weekly Barometers, then added six dedicated campaigns of 400 executions each.
This monitoring thus compares the same de-identified profile across multiple time scales, using a stable protocol. The analysis focuses on the level of the signal, its direction, its persistence, and any potential return to the initial regime.
An isolated variation is not enough to conclude that a drift has occurred. However, its repetition across multiple campaigns allows us to formulate a hypothesis of a regime shift, which must be confirmed through further observations.
⚠️ Longitudinal Monitoring Results Profile DEA9C5 (June 12, 2026)
The Barometer #4 measured a factual risk of 6.83% on the observed profile.
Subsequent longitudinal monitoring first showed an increase to 8.18%.
The signal then stabilized over four complete runs between 13.78% and 14.33%.
This trajectory suggests the establishment of a new factual risk plateau around 14%..
The result is more significant than an isolated spike, as it repeats across multiple full series.
It highlights the value of continuous monitoring between weekly snapshots.
For risk and compliance teams, this type of signal enables earlier triggering of targeted reviews.
For insurance purposes, it provides a useful temporal trace to document actual exposure and its evolution.
For finance departments, it helps focus human controls on areas where degradation becomes persistent.
NeoMundi thus transforms an invisible drift into actionable evidence, enabling better decisions, smarter resource prioritization, and a reduction in overall risk costs.
Toward Longitudinal AI Governance Reporting
Building on this monitoring protocol, NeoMundi is developing a longitudinal AI governance reporting framework designed to observe system behaviors over time, going far beyond one-off testing.
The project will gradually expand the scope of analysis to sensitive and operational domains, healthcare, legal, finance, stress tests, and business use cases, while cross-referencing signals of stability, semantic variation, and factual risk with indicators such as cost, token consumption, latency, information density, and human supervision effort.
The objective is to transform scattered technical measurements into a clear, actionable steering instrument for product, risk, compliance, finance, and executive teams.
This reporting will help answer concrete operational questions: Where is model behavior changing? What is the cost? Which use cases require reinforced controls? And where can the organization focus its resources for maximum impact?
NeoMundi is thus building a governance tool oriented toward decision-making, performance, and ROI, one that connects the real trajectories of AI systems to their operational, human, and economic consequences.
Methodology and public data
This Barometer tracks 12 de-identified profiles, each tested with 4 fixed questions repeated 100 times. In total: 4,800 executions, with 99.125% coverage.
The scores are measurement signals, not verdicts or rankings.
→ View the aggregated public data for AI Barometer #4
→ View the NeoMundi public baseline
→ View the Observatory’s methodology