The Factory Builds, the Evidence Admits: Anatomy of an Agent Certification

By Sam Wen, CEO & Founder of XInfer.AI — July 26, 2026

Sooner or later, someone across the table asks the question. A client's security team. An auditor. An acquirer's diligence engineer. A regulator. Or you, at two in the morning:

"Why is this agent allowed to do that?"

Not does it work — demos answer that, and demos are cheap. Not how good is it — benchmarks answer that, about someone else's test set. The question is why it is allowed: what, concretely, stands behind the decision to let a language model operate near your customers and your money, and would that reasoning survive being read back to you a year later.

We build AI agents that transact — chat, voice, phone, SMS, one deterministic core underneath all of it. In an earlier essay we argued that the language model should conduct the conversation while that core makes every decision. In a second, that the model which writes must never be the model that grades. This essay is the third act, and it asks what the first two were building toward: when an agent is done, what actually admits it to production?

Our answer is unglamorous. Not a launch review, not a leaderboard, not a sign-off meeting. A file.

A note on what kind of essay this is, because the subject demands it. The first two essays reported what we had built and measured; every claim had a receipt. This one goes further than the receipts. The system it describes is real and in production — but parts of what follows are extrapolation from the work we have done: conclusions the built system points to, worked out on paper, not yet running. I will mark the line as we cross it. An essay about certification that blurred the line between measured and extrapolated would fail its own bar.

Part I — The question a demo can't answer

Strip the ceremony away and "this agent may run in production" asserts four things:

1. Its authority is bounded. There is a known, finite list of decisions it can cause, and everything off that list is impossible — not discouraged, impossible.
2. Its inputs are trusted by origin. Every piece of data it acts on has a provenance, and provenance — not mere presence — determines what may be done with it.
3. Its words are measured. Whatever the model still says freely has been evaluated against evidence, by something that is not the model.
4. All of it replays. Anyone, later, can reconstruct exactly what was approved, against what evidence, and check the reasoning themselves.

Notice what's missing from that list: the builder's confidence. "We built it and it works" is the builder grading their own homework — the precise conflict of interest the second essay spent five parts constructing independence against, now at the scale of a whole agent instead of a single sentence. A model must not grade its own writing; a team cannot certify its own agent. Not because teams are dishonest, but because the incentives point one way and the weights, so to speak, are baked.

The first essay separated the decider from the narrator inside one request. The second separated the evaluator from the writer inside one build. Certification is the same cut, one level up: it separates the party that builds from the thing that admits. And the admitting party, in our view, should not be a committee. It should be the evidence.

Part II — The evidence file

What follows is built and measured — this is the file we can produce for our own agent today.

Authority is a list, not a mood. When our model submits an order, the request lands in one server-side function, and that function answers with one of five typed outcomes — order placed, contact needed, contact to confirm, cart empty, or a failure with a reason code. That list is the agent's authority over money. Nothing else can happen, because there is nothing else the core returns. This is the first thing that belongs in an evidence file, and the cheapest to check: the enumerated decisions. If you cannot write the list down, your agent does not have bounded authority; it has tendencies.

Seriousness is not the criterion for what goes in code. Determinism is. The temptation is to move everything "important" into the deterministic core. That's the wrong test. What moves into code is what has a code-time-knowable correct answer — thresholds, lookups, idempotency, required disclosures. Force a genuine judgment call into code and you either reduce it to a crude proxy or smuggle a model in through the back door, destroying the very property that made the core trustworthy. The right move, when a judgment is needed, is the one the second essay described: the model emits a typed verdict, and the core decides the action. Judgment measured by a model; consequence owned by code. The same split, applied recursively.

There is a corollary we now treat as a rule: an agent may make a decision only where the decision rule already exists in deployable form — a written policy, an existing validated system, a governed model. Where the rule exists nowhere — where the call is genuinely discretionary — the agent does not get to manufacture it, and neither does your prompt. The agent does the perception: it reads the messy input, extracts the structure, drafts the summary. A person, or captured logic, decides. This sounds restrictive; in practice it's liberating, because an enormous amount of valuable work was never automated not because the logic was missing but because the inputs arrived unstructured. The rules were always written down. They were gated on reading. Reading is exactly what these models are for.

Provenance, and evidence that is spent. The first essay covered this machinery in detail, so a summary: our core trusts data by where it came from — supplied this turn, established this session, on file from before — and each origin authorizes something different. Session evidence is a single-use token, consumed by the order it vouches for, so a second order can't ride the first one's proof. Submissions claim the cart atomically, so a model that retries a timeout replays the same order instead of minting a twin. Every one of those sentences is a unit test somewhere; that's what makes them evidence rather than architecture slides.

The gate is not a dashboard. Our judge — a different model from the writer, reading the same written standard — returns typed verdicts, quotes each claim verbatim, names the evidence path it checked against, and reports the claims it cleared alongside the ones it flagged. But the detail that matters for certification is what happens downstream of a verdict: grounding failures block the merge; quality is tracked as a trend. One is a gate, the other is a gauge, and confusing them is how eval programs die. A check that cannot fail the build is not a control. It's a dashboard with opinions.

Everything replays. Every judged run persists what it saw: the prompt, the evidence supplied, the verdict, the model and version. Prompt versions and knowledge content are hashed into cache keys, so no cached output can ever have come from an unknown prompt — edit the standard and everything that depended on it regenerates by design. Every prompt change carries a changelog entry: the case that forced it, the reason, the change, the judged verification. The second essay said that a verdict you cannot re-check is not an instrument but a mood. One level up, the same law: a certification you cannot replay is not a decision. It's an opinion.

Part III — The map, and the open column

This part is an audit, not a system — real work, done on paper, against standards we didn't write.

Confidence in your own architecture is worth little; the essays above are, after all, the builder talking. So recently we did the adversarial thing: we took the two patterns — core-decides, judge-measures — and put them in front of twenty-five requirements drawn from NIST's AI Risk Management Framework and the EU AI Act's high-risk articles. Frameworks written by people who have never read our prompts and owe us nothing.

The map came back in three colors: ten requirements covered, ten partial, and the rest open.

Covered clustered exactly where you'd predict, which is itself a useful check. Accuracy and truthfulness — the judge's whole existence. Robustness and reliability — idempotency, cross-channel consistency, retry-safety. Logging and traceability — typed outcomes are auditable by construction; you don't add logging to a decision that is already a value. Independent measurement — a calibrated evaluator that is not the writer. In NIST's vocabulary, the judge is a Measure function and the core is a Manage function, and the two patterns turn out to be those two boxes, implemented.

Partial is where architecture helps but doesn't finish: data governance, transparency, human oversight. The core makes oversight possible — a transparent, logged, explainable decision that a person can review is strictly better than a model output to rubber-stamp — but possible is not the same as done. Disclosure to the end user sits here too, and it's an instructive case. Every agent we ship introduces itself as an AI in its opening line — "I'm Kai, your AI assistant. How can I help you today?" — across chat, voice, and phone alike. But that sentence lives in a configurable greeting, not an enforced gate: a merchant could edit it away. Handled by default is not the same as guaranteed by construction, and a map that blurred those two would be flattering itself.

Then the open column, and here is the finding that reframed the exercise for us. The open items were: accountability and ownership. Governance of third-party and vendor models. Bias and fairness examination. Privacy and PII handling. Runtime monitoring and incident response.

Read that list twice. Every open item is organizational. Not one of them is a gate you can design. No typed outcome produces an accountable owner; no judge verdict staffs an incident channel at 3 a.m.; no cache key reviews a vendor. The architecture ran out precisely where the org chart begins — which is not a failure of the architecture. It is the discovery of where certification stops being an engineering problem. An audit that came back all green would have told us nothing except that we'd audited ourselves gently.

One more honest finding from the same exercise, because it surprised us: "high-stakes claim" and "high-risk system" are different axes. Under the EU AI Act, a commerce agent is limited-risk — its legal obligation is disclosure, not truthfulness. The Act tiers by domain, not by how expensive a false sentence is. Our guardrail work is justified commercially — by what a false promise costs a merchant — not because a statute demands it. Teams over-claim compliance pressure in both directions; the map keeps you honest about which force is actually pushing.

And the audit exposed a seam we had not named before. The judge works in development — it grades output to find where the writer's prompts leak, then stays out of production, where the deterministic gates stand watch. Which means: at runtime, nothing independently verifies the narration. The words the model speaks to a live customer pass regex and locked facts, and nothing else. For our risk profile today, that's a reasoned trade. But naming the seam tells you exactly what the next mechanism has to be — and that is where the receipts end.

Part IV — Extrapolations

The line is here. Everything below is extrapolation from the work above — worked out on paper, argued from the built system, not yet running. Our own judge would flag us if we let you read it any other way.

First extrapolation: gate the consequence, not the utterance. You cannot batch-check a sentence after a customer has heard it — a slow check on spoken words is monitoring, not a guardrail. But you can gate the consequence. Capture the irreversible step immediately and idempotently — the order lands, marked pending review. Then run the full-strength judge in the window between capture and fulfillment, where it has no latency budget at all: the entire transcript against the entire evidence file, all the time it wants. Deterministic code reads the verdict and releases, holds, or corrects — before anything ships, charges, or becomes real.

Payments people will recognize this instantly: it is authorize-then-capture with a fraud hold, applied to conversation. Fast cheap prevention at the utterance; slow accurate judgment at the commit; the irreversible step gated in between. The pattern has one precondition — a reversible window must exist. Instant digital delivery has none; pure advice has nothing downstream to gate; both fall back to strong inline prevention. Our core is idempotent and claims orders atomically today; the hold-window with an asynchronous judge deciding release is design. It is, we think, the single most underused idea in agent safety — mostly because the people who know the payments pattern and the people building agents have not been the same people.

Second extrapolation: the admissions office. Scale the argument of Part I up from one agent to a catalog of them — the "agent factory" every large organization is now standing up — and the logic forces a structure. If a certification is a file, and the builder cannot self-attest it, then somewhere there is a party whose job is to read the file and say no. Separate from the builders. Owning the bar, not the backlog.

That party's product is not agents. It is the bar — and the credibility of the bar is measured by exactly one number: the rejection rate. A catalog with no rejection rate is not a control. It is a spreadsheet with better branding. Every organization that stands up an agent program announces a catalog; almost none can tell you what their bar has refused. Twelve months later there are four hundred entries, zero rejections, and no claim anyone outside the team believes.

The same file does one more job: it is how reuse gets enforced rather than encouraged. Once an agent's pattern is certified, the file is the reference — and the second team that needs the same capability doesn't get a second certification for a copy; they get pointed at the first one. The catalog becomes the place where duplicate work goes to be refused. We run this logic at merchant scale — one certified checkout protocol, reused across every channel rather than re-certified per channel. At enterprise scale we have not run it, and this essay won't pretend otherwise. The shape is extrapolation. The forces it answers — builders grading themselves, catalogs that only ever grow — are not.

Part V — What generalizes

The three essays in this series are one idea wearing three sizes. At every layer, one party proposes, and a different party — with different incentives, reading the same written standard — decides:

If you are deciding what "certified" should mean for agents — yours, or a catalog of them — here is the shape of the argument, portable:

1. Certification is a file, not a ceremony. Enumerated authority, provenance rules, measured words, replayable runs. If you cannot replay the decision, you have an opinion with a date on it.
2. Check authority first — it's the cheapest gate. The list of decisions an agent can cause is knowable before a single behavioral test runs. No list, no bounded authority.
3. Determinism, not seriousness, decides what lives in code. And where judgment is genuinely needed: the model emits a typed verdict, code owns the action.
4. An agent decides only where the rule is captured. Everywhere else it reads, extracts, and drafts — perception — while captured logic or a person decides. Most of the value is in the reading anyway.
5. Audit against a standard you didn't write. Your own rubric will be kind to you. The open column — what your architecture cannot certify — is the most valuable output, because it is a precise list of what the organization must supply instead.
6. Gate the consequence, not the utterance. Where a reversible window exists, capture fast, judge slow, and let code release the hold. Where it doesn't, prevention is all there is — know which case you're in.
7. Separate the builder from the bar. And publish the rejection rate. It is the only number that proves the bar exists.

One honest caveat, in the house style: the file certifies the agent. It cannot certify the organization around it — no artifact we know how to construct measures whether an accountable owner will actually show up when the incident channel fires. That column stays open, and pretending a gate design closes it would be exactly the kind of fluent, unsupported claim this whole series exists to catch.

The extrapolations above are unbuilt, and we've said so. That is not a hedge; it is a roadmap with witnesses. When the hold-window ships, it will get its own essay — receipts first.