An Enterprise-Grade Architecture for AI That Doesn't Hallucinate

In order to build the Neutron Enterprise framework, we research and develop production-grade techniques for deploying agentic AI in the toughest regulated enterprise environments. Eliminating hallucination is one of the areas we focus on most, and we’ve built several mechanisms toward that goal. This post is about one of them: the grounding gate.

The grounding gate is a deterministic architecture that checks every drafted answer against the data the system actually fetched during that request, and withholds the whole answer if any fact in it cannot be traced to that data. It is built as a control, not a lower average: no second model acting as a judge, no embedding similarity, no probability threshold, because none of those produce a decision you can show to a regulator and reproduce on demand. The write-up is detailed enough that you could build something similar yourself, and honest enough to show what it took to get ours right.

Why it matters: a large language model is a fluent generator with no internal notion of whether a statement is backed by data. Left alone, it will produce a number, a name, or a relationship that is plausible but was never in anything it was shown. In a consumer chatbot that costs you an embarrassing screenshot. In a regulated enterprise it is a liability event with consequences that dwarf the value of the answer.

What separates these industries is that they are required to prove control over specific answers after the fact:

• Financial services (SR 11-7 model-risk management, SEC/FINRA recordkeeping): every material figure surfaced to a decision-maker must be traceable to a source, with a demonstrable control guaranteeing it.
• Healthcare and life sciences (FDA, HIPAA): an assistant that invents a dosage or an adverse-event count is a patient-safety failure.
• Legal and contract review: misstating a clause or a party is malpractice-adjacent.
• Defense, intelligence, critical infrastructure: provenance and the ability to refuse are mission requirements, because a confidently wrong summary drives wrong decisions.
• Government and public sector: answers are FOIA-exposed, and “the AI made it up” is not an acceptable incident report.

“Our model hallucinates 40 percent less” is a statement about an average. It has nothing to say to an examiner asking about one specific answer given to one specific analyst on one specific day. The claim these buyers need is structural: the platform makes it impossible to surface a fact the system did not verify this turn, and here is the tamper-evident record showing exactly which rules judged that answer. That is a claim about a control, and building it as a control shaped every decision in this design.

The rest of the post is in two parts. First the architecture at a system level, which is what a technical decision-maker needs in order to evaluate the approach. Then the implementation in depth: how the grounding corpus gets built, why checking that a number exists in the data is nowhere near enough, how spelled-out numbers and recombined names get caught, and how a strictness knob can be configurable without ever being able to weaken a fact check. Along the way I’ll cover two production incidents and the three defects our adversarial review pipeline caught in the fix itself, because a team that only publishes its successes is hiding its engineering process. The failures, and the machinery that caught them before a customer could, are the evidence that the process works.

The architecture at a system level

Every factual read in the system goes through a governed query template that executes server-side and returns a structured object we call a verified answer: columns, rows, a citation per row, and an envelope recording how many rows were returned versus how many exist in total. That object is the ground truth for the turn.

Take the simple read case first. A claims adjuster asks “what’s the status of claim 48213?” The system runs a governed query against the claims database and gets back exactly one row (claimant, status, amount, last-updated date) with a citation pointing to that record. There is nothing to check here: the verified answer itself is the response, straight from the source system, with no model narration in between.

The interesting path is the grounded agent. Take a collections workflow: an agent is asked to chase overdue invoices. It runs a verified read against the ERP for accounts more than 60 days past due, gets back rows for a dozen accounts with balances, due dates, and invoice numbers, and then has to act on that data, maybe summarize it for the finance team, maybe draft a follow-up email to each customer citing their specific balance. Turning rows into a summary or an email means a model deciding which reads to run and narrating over the results in ordinary English, and narration is exactly where a wrong attribution can slip in: the right dollar figure written under the wrong customer’s name. That narration is the draft, and the draft is what the gate judges before it reaches the finance team or a customer’s inbox.

The request path from draft to user works like this. The runtime builds a grounding corpus from this turn’s fetched data and nothing else: no prior turns, no world knowledge, no model memory. A deterministic check_grounding function then classifies every claim in the draft against that corpus. Every numeral, every entity identifier, every citation marker, and every word of prose has to ground through something the corpus can account for. The findings pass through a per-deployment strictness policy, apply_strictness, which decides pass or withhold based on the deployment’s assurance tier. A passing draft is emitted with a provenance stamp recording exactly which rules and sources judged it. A failing draft is withheld in full: the runtime may route it through an optional bounded rescue, a single revision attempt whose output faces the same gate, and if no candidate passes, the user receives a deterministic fallback built from the verified data itself, a short notice plus the actual fetched tables with their citations.

The design takes a deliberate position here. A second model judging the first (LLM-as-judge) or an embedding-similarity score with a tuned threshold both replace one probabilistic component with another, which means neither can anchor the compliance claim above. This architecture is built on the opposite stance, expressed as three control properties that drove everything below:

1. Deterministic. No second LLM acting as a judge, no embedding similarity, no probability threshold. The same draft plus the same fetched data always produces the same verdict, so the control can be shown to a regulator and pinned by a test.
2. Fail-closed. When in doubt, refuse: zero fetches means nothing is speakable, an unparseable numeral refuses, and a malformed fetched result contributes zero facts.
3. Privacy-preserving. A blocked answer’s audit record carries only a reason class and a count, never the offending text, so the control cannot become a data-exfiltration channel.

Everything from here down is the implementation of those three properties, including the places where our first implementation got them wrong.

Inside the runtime turn

The gate runs as a post-step inside the runtime turn, after the model produces its draft and before anything is persisted or streamed to the client:

model draft ──► build grounding corpus (this turn's fetches only)
            ──► check_grounding(draft, corpus, prompt) ──► verdict
            ──► apply_strictness(verdict, deployment tier)
                 ├─ pass ► emit draft, stamp provenance
                 └─ fail ► audit one counts-only block
                           ► withhold the draft entirely
                           ► optional bounded rescue, re-checked by the same gate
                           ► else emit deterministic fallback (notice + verified tables)

One implementation detail does a surprising amount of work: the gate function has exactly one return statement, and every value that can ever be emitted either already passed the check or is re-checked before assignment. That single-emit discipline is what makes the audit trustworthy, because no code path emits ungoverned text.

The grounding corpus: the universe of things an answer may say

The corpus is built fresh each turn from that turn’s fetched verified answers and nothing else. If zero fetches happened, the corpus is empty and every content token in the draft refuses. In grounded mode, not fetching means not speaking.

Construction is itself fail-closed. Only results carrying the exact verified-answer shape contribute facts; a blocked tool result, a malformed mapping, or a row whose length disagrees with the column list contributes nothing at all.

From each verified answer, the corpus extracts five kinds of material:

1. Phrases: per-row multi-token cell values; a stated multi-word value must appear as a real row phrase, not a bag of individually known words.
2. Tokens: the normalized single-token vocabulary of all cells, for grounding ordinary prose.
3. Entity IRIs: every entity identifier in the fetched cells. An identifier in the draft must be one that was actually fetched.
4. Numbers: a map from each canonical numeral to the fact records that produced it.
5. Citation count: how many citations the answer carries, so a [7] marker can be range-checked against reality.

Two hardening rules are baked into extraction because red-teaming found the holes early. First, any cell whose column name matches one of the query’s own argument keys is dropped, because a column that merely echoes caller input is not data, and letting it ground would let a caller launder arbitrary strings into speakable facts. Second, a trailing percent sign or currency symbol is meaning-bearing during numeral canonicalization: “50%” canonicalizes to the distinct token 50%, never to plain 50. Since the gate refuses to do arithmetic, a percentage grounds only if that exact percentage is itself a fetched cell. Otherwise the model computed it, and computed values always refuse.

Identity anchoring: why token existence is not enough

Here is the failure mode that makes naive grounding checks nearly worthless. Consider the sentence “Initech has 3 open orders.” Suppose 3 is a perfectly real number in the fetched data, but it belongs to Acme’s row, and Initech has none. A checker that asks “does this number appear anywhere in the data?” waves the sentence through. The number is real. It is attached to the wrong company, and that misattribution is the dangerous hallucination in practice, because it survives spot checks: anyone who greps the source data for “3” finds it.

So in this gate, a numeral does not ground by existing in the corpus. It grounds only when the sentence that states it also names the identity of the row that produced it.

Mechanically: for each numeric column kept in the corpus, compute per row the complete token set of a single non-numeric, non-identifier sibling cell, typically the name column. That token set is the row’s identity anchor. Discard any anchor that is a subset of another row’s anchor in the same column, because an ambiguous anchor cannot disambiguate anything. A number fact then grounds only if the draft sentence’s grounded prose tokens include that row’s anchor. The effect is to change what the gate certifies, from “the model said a number that exists” to “the model attributed a number to the entity the data attributes it to,” which is the property an auditor actually cares about.

Numbers in depth

Numbers are where confident hallucination costs the most, so they get the most machinery.

Raw cell numerals and ordinals ground through the identity-anchor rule above (“the 3rd account” is handled like “3”).

Envelope counts ground separately. A verified read can be truncated by a row cap or a token budget, and honest truncation must remain speakable. The envelope fields (returned rows, total rows, row limit) each mint a number fact, so “showing 3 of 5,000,000 accounts” passes when the envelope says so, while an invented “there are exactly 5,000,000 accounts” refuses when no envelope or cell asserts it. Each field is parsed fail-closed on its own: a stated returned-row count that contradicts the actual row array length is muted, as is a total smaller than the returned count (a nonsensical “returned 3 of 1”), and a bad field silences only that one numeral, never the whole answer.

Derived aggregates always refuse. If the model sums, averages, or subtracts its way to a total that appears in no cell and no envelope, the gate cannot verify it without performing arbitrary arithmetic, and it deliberately does not, because recomputing would put the gate in the business of doing math the model may have set up wrong. The sanctioned pattern is to move aggregation server-side: a governed COUNT or aggregate template computes the total under the same controls as everything else and returns it as a cell, so the model reads a verified total instead of computing an unverifiable one.

Spelled-out numbers route through the same machinery via a closed lexicon of 56 number words: “one” derives to “1” and grounds only if a real 1-cell with the right identity anchor exists. Spelled numbers are never exempted just because the user’s prompt contained the same word (prompt vocabulary is exempt for prose, never for numerals, or a prompt-echoed “twenty” could launder a false count), and magnitude words like “dozen” and “million” always refuse, since they assert no exact value the corpus could confirm.

Compound spelled numbers are canonicalized as one numeral: an adjacent run of number words (“twenty one”) either resolves to a single grounded value or refuses as one unparseable numeral at the head of the run. The first implementation validated compounds component-wise, which turns out to be a fact-fabrication bypass; more on that in the bugs section.

Phrase recombination: the anti-Frankenstein rule

Token-level grounding has a second hole beyond misattributed numbers. A model can stitch “Thomas More” from one row and “More Industries” from another into “Thomas More Industries,” an entity that exists nowhere. Every token in that phrase is real corpus vocabulary, so a per-token check passes it.

The gate closes this by tracking runs of consecutive corpus tokens and, when a run ends, checking that it corresponds to an actual corpus phrase. A run that recombines fragments across rows is flagged as phrase recombination, a fact-level finding, because it fabricates an identity. One subtlety worth copying: corpus tokens take precedence over allowlist words inside a run, so a connective that doubles as corpus vocabulary cannot break a run in two and let a recombined identity slip past the phrase check in halves.

Classifying the draft

With the corpus built, the check splits the draft into sentences, tokenizes them, and classifies every event. Each finding lands in one of a small set of classes: ungrounded numeral, unparseable numeral, ungrounded entity IRI, out-of-range citation, phrase recombination, and a few vocabulary classes for prose tokens that ground through nothing.

Every prose token has exactly three ways to ground: it is real cell vocabulary from the corpus, it is a word the user’s own prompt used (vocabulary-exempt, never numeral-exempt), or it is on the connective allowlist. A token that grounds through none of these is an ungrounded-vocabulary finding, with one escalation rule that matters: if its sentence also carries a numeral, an IRI, a citation, or a number word, the finding escalates to a fact-adjacent class, because a fabricated modifier sitting next to a fact is exactly where damage happens. The split between fact classes and vocabulary classes is the hinge for the strictness section below.

The connective allowlist, and why not a semantic model

A gate that accepts only corpus tokens and prompt tokens rejects all natural English. We learned that the hard way, the first time the gate met a real model. We were validating an unrelated feature against a 32-billion-parameter model on a GPU rig. It had fetched a set of accounts through a governed query and narrated the result in ordinary English:

“You are cleared to see the following accounts and their respective open orders: Bluepeak Dynamics LLC: 1 open order … Additionally, there are 25 more accounts, each with 1 open order, making a total of 28 accounts.”

Every number in that answer was right and every name traced to a fetched row. The gate withheld the entire narration anyway. When we replayed the gate locally against that exact text, the findings had nothing to do with facts. Five ordinary connective words (additionally, following, making, more, respective) were missing from the gate’s 135-word allowlist of permitted non-factual vocabulary, and the gate treats any token it cannot account for as a reason to refuse. No production model narrates in telegraphese, and a gate that requires telegraphese is unusable on the grounded-agent path.

The fix is a positive allowlist of non-factual English function words: conjunctions, prepositions, determiners, quantifiers, auxiliary and gerund verbs. Ours grew from the original 135 words to a 320-word core (including the five words from the live failure) plus an 87-word extended list of relational and quantified phrasing that only the more permissive tier consults, plus the 56-word number lexicon, which derives numerals and never exempts anything.

The obvious objection: why a word list and not a learned entailment model that asks “does the data support this sentence?” Because that would reintroduce the exact component this design exists to eliminate. An NLI judge is probabilistic, so identical inputs can produce different verdicts; it is unauditable, since “why did it pass?” has no inspectable answer; and it is itself hallucination-prone. An allowlist is a data file you can read, diff, hash, and test.

The allowlist’s safety invariant is that it may only ever exempt non-identity, non-numeric vocabulary. A careless future addition can loosen prose, and nothing else. Hygiene tests run against the final resource files to enforce this: the current core must remain a superset of the frozen original, the lists must be pairwise disjoint, every entry must normalize cleanly at import, and a red-team fixture of identity-shaped words (company-name fragments, IRI scheme parts) must appear on none of them. The allowlist bytes are also hashed into the gate’s version string, so an edit cannot ship silently.

Configurable strictness: one gate, per-deployment posture

Different deployments sit at different points on the assurance-versus-usability curve: a defense deployment may want every finding, including a stray adverb, to withhold, while a general enterprise wants natural narration to pass with every fact still verified. So strictness is a knob, with two non-negotiable properties.

First, it is a per-deployment setting, never a per-request one, because a caller-selectable tier is a self-service assurance downgrade.

Second, and this is the load-bearing law of the whole design: the knob can only ever move non-factual vocabulary. The enforcement matrix lives in one place and reads, in essence:

FACT_CLASSES = ALL_CLASSES - {"vocab-ungrounded", "vocab-extended"}

ENFORCEMENT = {
    StrictnessTier.STRICT:   ALL_CLASSES,    # everything withholds
    StrictnessTier.STANDARD: FACT_CLASSES,   # only fact classes withhold
}

Every fact class, plus the fact-adjacent escalation, withholds unconditionally at every tier. Only pure-vocabulary findings change behavior: enforced at STRICT, advisory at STANDARD, where advisory means recorded in the audit as a class and count, never dropped, never blocking. A guardian test iterates every member of the tier enum and fails CI if any tier, including one added years from now, drops a fact class from enforcement. That single test lets you tell a regulated buyer that no configuration, present or future, can make a fabricated fact speakable.

Strictness (the posture) is orthogonal to the rulebook (which pinned set of rules and allowlist bytes is in force). We keep the original rulebook frozen, its version derived from a hash of its exact resource bytes, so the highest-assurance deployments can pin the historical behavior byte-for-byte; the current rulebook hashes its own rules and resources the same way. Both the tier and the rulebook version are stamped onto every answer’s provenance record, so any answer can be re-judged later under exactly the rules that judged it the first time.

The rescue pass: salvaging near-misses without weakening the gate

A capable model often produces a draft that is one derived total or one stray connective away from passing. Withholding the whole answer is safe but harsh, so the runtime supports a bounded, single-pass rescue: ask the model once for a revised draft, then re-check the candidate with the same gate before it can be emitted. The re-check is the entire safety argument; the rescue cannot weaken anything, because its output faces the identical judge.

The information flow is deliberately asymmetric. On a failure, the rescue receives the question, the verified tables, and the reason classes only, never the withheld draft text, so a failed draft cannot leak its fabrications into the retry. Every rescue invocation emits a counts-only audit event (phase, whether the candidate was adopted, an error class if one occurred), and both the rescue-used and fallback outcomes are stamped on provenance.

What adversarial review caught in the fix itself

We treat a fix to control-plane code as untrusted output, exactly the way the gate treats a draft. The rebuild went out as a sequence of small, dependency-ordered changes, each of which had to survive an adversarial review pipeline before merge: an automated reviewer pass, then an independent cross-review by a different model. The original defects had shipped green through a conventional test suite, so reviewing the fix any less aggressively than the gate reviews a draft would have repeated the mistake. The pipeline caught a genuine fact-fabrication bypass sitting inside the fix meant to eliminate exactly that class of error.

A fact-fabrication bypass in compound spelled numbers. The new number-word handling validated “twenty one” component-wise: “twenty” derived to 20 and grounded, “one” derived to 1 and grounded, each against whatever cells happened to contain those values anywhere in the corpus. Which means a fabricated 21 passes whenever an unrelated 20 and an unrelated 1 both exist somewhere in the data. This is precisely the class of error the gate exists to stop, sitting inside the gate’s own fix. The repair treats an adjacent run of number words as one indivisible numeral, refusing at the head of the run if it cannot resolve as a single grounded value, pinned by a test written to fail first, with the number-word map frozen at import.

It wasn’t found by intuition or by the existing suite. It surfaced because our process assumes every fix is wrong until something adversarial has failed to break it. That assumption costs a review cycle, and it caught a fact-fabrication bypass sitting inside the anti-fabrication fix itself.

Refusing usefully

A gate that returns a bare error on withhold trains users to hate it. When this gate withholds, it emits a deterministic fallback: a short notice that the drafted narration contained content that could not be verified against this turn’s fetched data, followed by the verified tables themselves, real rows with real citations. The user still gets the source-backed data; what they lose is the model’s unverifiable prose over it.

One quarantine rule matters for multi-turn systems: only the fallback exchange, stamped with an unforgeable gate marker, is persisted to conversation history. The failed draft is never written anywhere the next turn’s context could pick it up, so a withheld fabrication cannot launder itself into a later answer.

Proving it after the fact

The pieces that make this auditable end to end:

• Counts-only findings. A finding is a reason class and a count; the draft text never leaves the gate module, so the audit contains no sensitive content and is safe to retain indefinitely.
• A hash-chained audit. Every block, every advisory, and every rescue event is a fail-closed record in a tamper-evident chain. If the audit write fails, the run dies rather than proceeding unaudited.
• Never fake. The gate never edits a draft to make it pass; a gate that rewrites answers is authoring text no one checked. It emits the model’s own certified draft, a rescue candidate that independently re-passed the same gate, or the deterministic fallback. There is no “mute the bad word and ship the rest” path.
• Provenance on every answer. Each governed answer records the fetches it grounded against (template, version, row counts, truncation flags, never row text), the rulebook version, the strictness tier, and the fallback and rescue flags. When an examiner asks “prove this figure was verified,” the answer is a reproducible record of exactly which rules and sources judged exactly that answer.

Testing it like an attacker

We wrote the acceptance suite as an attack plan before writing the implementation: a red-team matrix that asserts WITHHELD across every combination of tier and legal rulebook for each of these:

• a fabricated name attached to a true numeral (the Initech case, with the count being real);
• a fabricated name with a fabricated numeral;
• a wrong count lifted from a sibling row;
• a numeral leaked from data the requesting principal cannot see;
• an invented entity IRI;
• an out-of-range citation marker;
• spelled counts, including “one open order” when the cell says 3;
• a prompt that itself contains the spelled number under test;
• name-fragment recombination, including through a newly added allowlist word.

And the matrix asserts PASS at the permissive tier on the exact captured narration from the over-strict withhold incident described earlier. That transcript is now a permanent regression fixture.

In summary, the key principles: ground numbers to row identity, refuse what you cannot verify structurally, keep the vocabulary tolerance in inspectable data rather than a model, make the strictness knob provably unable to touch facts, and run the result against a real model’s real prose before you believe any of your tests.

Neutron Enterprise

This anti-hallucination gate is just one example from the Neutron Enterprise framework, governed AI for organizations where “the model made it up” ends up in an incident report. Verified server-side reads, this grounding gate on every narrated answer, per-deployment strictness, and a hash-chained audit trail with rulebook versions stamped on every response. If you want to learn more about the capabilities of Neutron Enterprise, and get help deploying something like this in your organization, reach out.