Independent Audit Report by Pape Malick DIOP, Data Scientist and Machine Learning Researcher, June 10, 2026.
At NeoMundi, we believe that the governance of generative systems cannot rely on a single signal. This is why we commissioned an independent external exploratory audit of our runtime signals: G-Score, ∆G, FLAG, regime, and benchmark accuracy.
Pape Malick DIOP analyzed the exports provided by the ControlTower tool from an observability and governance (OBS/GOV) perspective. Here is what his study reveals. The full report is available as a PDF at the bottom of this article.
Key Findings of the Exploratory Audit of NeoMundi Real-Time Signals
Analysis of G-Score, ∆G, FLAG, Regime, and Benchmark Accuracy
The report confirms several important insights:
- The G-Score primarily measures real-time stability, not factual truth.
The correlation between the G-Score and benchmark accuracy is very low (approximately 0.10). A high score does not guarantee a correct answer. - The phenomenon of « deceptive stability » is real and must be taken seriously.
Some providers display very high G-Scores (up to an average of 0.9231) with near 100% saturation, while showing significantly lower benchmark accuracy than other providers with similar G-Scores. - FLAG, DROP, and ∆G are useful alert signals, but not automatic proof of error. They help identify zones of tension or local breakdown.
- The STABLE regime is very dominant, which limits its discriminatory power when used alone.
Providers with very similar average G-Scores can exhibit very different levels of accuracy. A stable behavior does not necessarily mean a correct answer.
Proportion of cases where G ≥ 0.90 and the response is incorrect according to the benchmark. Provider P-003 particularly illustrates this risk.
How to Interpret NeoMundi Signals in Practice?
A single signal is never enough to draw a conclusion.
Like a car dashboard, multiple indicators must be cross-referenced to understand what is really happening.
Reference Grid – Signal-by-Signal
InterpretationFor each observed signal: its interpretation, main risk, and recommended action. No signal should be read in isolation.
| Observed Signal | Interpretation | Main Risk | Recommended Action |
|---|---|---|---|
| High G-Score alone | Stable, regular, or coherent runtime generation. | Possible confusion between stability and factual truth. | Never use the G-Score alone to validate a response. Cross-reference with accuracy or a factual layer. |
| High G-Score + Low Accuracy | Deceptive Stability: the model produces a stable but incorrect response. | High apparent confidence despite a factual error. | Trigger external factual validation or enhanced review. |
| Low or unstable G-Score | Signal of runtime fragility, irregularity, or less stable behavior. | Output potentially sensitive to context, model, or prompt. | Examine the query, provider, and associated variations. |
| High ∆G or DROP profile | Strong signal variation, local breakdown, or tension zone. | Runtime transition, local instability, or degradation. | Classify as priority attention zone and analyze with FLAG, regime, and accuracy. |
| FLAG | Runtime alert indicating that a response deserves verification. | Possible over-interpretation as an automatic error. | Use as an escalation or enhanced verification signal, not as systematic rejection. |
| FLAG + DROP | Stronger runtime risk zone, combining alert and signal breakdown. | Increased risk of instability or problematic behavior. | Prioritize analysis, consider human review or a second validation pass. |
| STABLE regime alone | Synthetic context indicating an overall stable state. | Limited discriminatory power when STABLE is dominant. | Never use the regime alone; cross-reference with G-Score, ∆G, FLAG, and accuracy. |
| ALLOW + Low Accuracy | Case not detected by the runtime signal. | Potential false negative in the governance logic. | Strengthen the factual or benchmark layer, especially offline. |
| High G-Score saturation | Concentration of the score around an observed ceiling. | Ambiguity between real stability and saturation effect. | Test robustness across multiple providers, benchmarks, prompts, and repetitions. |
Grammar of Rules – Actionable Interpretation Rules (R1–R10)
The same logic, expressed as named and citable rules. To operationalize an OBS/GOV reading and, later, configurable thresholds.
| Rule | Condition | Interpretation | Recommended Action |
|---|---|---|---|
| R1 | High G-Score + Low Accuracy | Deceptive Stability. | Add external factual validation. |
| R2 | High G-Score alone | Runtime stability, not proof of truth. | Never use G-Score alone for validation. |
| R3 | FLAG + DROP | Priority runtime risk zone. | Escalation or human review. |
| R4 | Low G-Score + High ∆G | Potential breakdown or tension. | Investigate as transition zone. |
| R5 | FLAG + Correct Response | Alert without direct factual error. | Read as attention, not rejection. |
| R6 | STABLE regime alone | Insufficient synthetic signal. | Cross-reference with G-Score, ∆G, FLAG and accuracy. |
| R7 | Stable G-Score + Low FLAG + Low Accuracy | Stable but factually weak profile. | Add factual layer. |
| R8 | High G-Score saturation | Strong stability or ceiling effect. | Analyze variability, ∆G, FLAG/DROP. |
| R9 | High G-Score variability between providers | Sensitivity to model or context. | Use as priority test case. |
| R10 | Single signal used alone | Insufficient decision base. | Use multi-signal reading. |
Main Recommendations from the Audit
Pape Malick Diop proposes a clear multi-signal interpretation grammar:
- Never use the G-Score alone to validate a response.
- Always cross-reference at minimum: G-Score + ∆G + FLAG + regime + benchmark accuracy.
- Use FLAG + DROP as escalation or enhanced verification signals.
- Consider the STABLE regime as a global context, not an isolated trigger.
The author concludes that NeoMundi signals are readable and usable for a first OBS/GOV-ready integration, provided they are combined and contextualized.
Why We Publish This Audit
We believe that transparency regarding the strengths and limitations of our signals is an integral part of our approach. This independent audit helps us:
- Better communicate what the G-Score actually measures (runtime stability vs. factual accuracy).
- Refine our governance rules.
- Continue improving the robustness of our signals on broader benchmarks and varied contexts.
We warmly thank Pape Malick Diop for the quality and clarity of his work.
Download the complete report
The complete redacted methodological review is available in PDF, including all reasoning, formulas, and limitations.
Pape Malick Diop
NeoMundi Research Contributor · Methodological Reviewer
Machine Learning, Uncertainty & Analysis of AI Runtime Signals
Pape Malick Diop is a data scientist and machine learning researcher specialized in statistics, probabilistic reasoning, model robustness, and uncertainty analysis.
Within the Observatory, he contributes as a methodological reviewer, with critical analysis of the runtime signals produced by ControlTower: G-Score, ΔG, informational density, behavioral regimes, FLAG signals, and associated indicators.
His contribution aims to evaluate the readability, consistency, and methodological limitations of these signals, particularly when used to observe transitions between runtime coherence, tension, drift, or behavioral instability.
He notably conducted an exploratory audit of NeoMundi’s runtime signals, contributing to strengthening scientific caution, documentation of limitations, and the robustness of future analyses of the Observatory.
Mission : Methodological review of runtime signals, analysis of behavioral regimes, exploratory qualification of NeoMundi indicators, and identification of interpretation limits.
Profile : [LinkedIn]