The Problem: Plausible Lies
Ask a general-purpose AI assistant about a router you manage. Give it a hostname, maybe an IP range. It will happily generate output that looks like a real show ip ospf neighbor response — complete with neighbor IDs, interface names, dead timers, and state values. Every field will be formatted correctly. The OSPF states will say FULL. The timers will look reasonable. And none of it will be real.
This is not a theoretical concern. In infrastructure operations, fabricated data creates specific, measurable danger:
- Masked failures. A fabricated “OSPF neighbors FULL” response hides a real adjacency failure. The operator sees a clean topology when in reality a branch office has been partitioned for hours.
- Phantom policies. An invented “26 active policies” response on a FortiGate hides a misconfigured firewall. The operator believes segmentation is enforced when it is not.
- False compliance. A generated compliance report that shows “PASS” across the board satisfies no auditor and protects no network. Worse, it creates false confidence that delays real remediation.
- Invisible outages. A fabricated interface status of “up/up” on a WAN link masks a circuit that has been down since 3 AM. The NOC never gets paged because the AI said everything was fine.
Key insight: The danger of AI hallucination in infrastructure is not that operators will blindly trust fake data. It is that fake data looks exactly like real data. A fabricated show running-config snippet is indistinguishable from a real one unless you SSH into the device and check. The whole point of an AI operations tool is to save you from having to do that manually. If you cannot trust the output, the tool has negative value.
What We Tried First (And Why It Failed)
We did not start with a compiled C binary. We started where everyone starts: trying to solve the problem in Python, in the same process as the AI.
Prompt engineering. We injected anti-fabrication directives into every system prompt — five hardcoded rules telling the AI to never fabricate data, to always use provenance tags, to report failures honestly. This worked most of the time. But “most of the time” is not acceptable when a false positive means an engineer misses a real outage. The AI would occasionally find ways to generate plausible output that technically didn’t violate the letter of the rules while violating their intent.
Data envelopes. We wrapped collector output in structured metadata — source tags, timestamps, collection methods. The AI was instructed to only reference data inside these envelopes. Better, but still operating in the same text space. The AI generates text. The envelopes are text. The boundary between “real data” and “AI-generated text” was enforcement by convention, not by architecture.
Validation layers. We added post-processing checks that tried to detect fabricated output by comparing it against known device inventories and expected formats. This caught some fabrications. It missed others. The AI is very good at generating output that matches expected formats — that is literally what it is trained to do.
After two months of these approaches, we realized the fundamental flaw: we were trying to solve a text problem with more text. The AI generates text. Real device output is text. Both exist in the same Python process, the same memory space, the same context window. No amount of prompt engineering or validation can create a hard boundary between them. The boundary has to be architectural — enforced by the operating system, not by the language model.
Our Solution: Hardware-Level Trust
We built tli-executor — a compiled C binary that handles all communication with infrastructure devices. It runs as a separate process. The AI platform (Python) communicates with it over a Unix domain socket. The AI cannot access the executor’s memory, its keys, or its internal state. The operating system enforces this separation — process isolation is not a convention, it is a kernel guarantee.
How It Works
When the AI platform needs to execute a command on a device, it sends a request to tli-executor over the Unix socket. The executor:
- Opens an SSH connection to the target device
- Executes the requested command
- Captures the raw output
- Computes an HMAC-SHA256 signature over the output using a 256-bit secret key
- Returns the output + signature to the Python platform
The Python platform verifies the HMAC signature before presenting any output to users. If the signature is valid, the output is tagged [VERIFIED]. If the signature is missing or invalid, the output is tagged [UNVERIFIED] and flagged for review.
The Key
The HMAC secret key is generated by the C binary at startup using a cryptographically secure random number generator. It exists only in the binary’s volatile memory. It is:
- Never written to disk — no config file, no key store, no environment variable
- Never exposed via API — no endpoint, no debug output, no log entry
- Never in the Python process — the AI platform can verify signatures but cannot generate them
- Never in the AI’s context — the model has no concept that the key exists, let alone what it is
When the executor process stops, the key is gone. Next startup generates a new one. The key has no persistence, no backup, no recovery path. This is by design.
The Architecture
The critical insight is the process boundary. The AI platform and the executor are separate processes with separate memory spaces. The Python process can send commands to the executor and receive signed responses, but it cannot read the executor’s memory to extract the signing key. This is not a software convention — it is enforced by the Linux kernel’s virtual memory system.
What This Means
[UNREACHABLE] with the error details. The failure itself is authenticated. The AI cannot hide a failure by generating fake success data.
Verified Output
Every piece of output now carries one of three trust states:
There is no scenario where the platform presents fabricated data as real. If the HMAC does not verify, the output does not get the [VERIFIED] tag. If there is no data at all, the platform says “I have no data for this device” instead of inventing an answer.
Side-by-Side: Three Approaches
The difference becomes immediately visible when a device goes offline. Here is what happens when an operator asks about router R3 and R3 is currently unreachable:
The left column fabricates data. The middle column tries to prevent fabrication with a prompt — and fails. The right column makes fabrication cryptographically impossible. The operator gets the truth: R3 is unreachable, here is the error, here is what to check.
Anti-Fabrication in Action
Here is a complete interaction showing how HMAC-verified data flows through a real session:
Notice what changed from our earlier architecture: Every response now carries an HMAC verification status. The [VERIFIED] tag is not cosmetic — it represents a cryptographic check that passed. Even the failure response for R3 is signed, proving that the failure was reported by the executor (not fabricated by the AI). The system did not generate fake data, did not silently skip R3, and did not say “everything looks good.” It reported exactly what it knew, cryptographically verified.
The Evolution: From Prompts to Proofs
Our anti-fabrication architecture evolved through three distinct phases:
Phase 1 and Phase 2 are still active — the provenance tags and prompt directives remain in place as defense in depth. But they are no longer the primary enforcement mechanism. The cryptographic layer is. If every other safeguard fails, the HMAC check catches it.
Why This Matters
Trust in Infrastructure Tools
Infrastructure tools only have value if operators trust them. A monitoring dashboard that sometimes shows false data trains operators to ignore it — which is worse than having no dashboard at all. The HMAC verification layer is not a feature of the platform. It is the foundation that makes every other feature trustworthy. Firewall management, network diagnostics, compliance audits — none of these capabilities matter if the underlying data might be fabricated.
Compliance and Audit Trails
Regulatory frameworks like NIST 800-53, HIPAA, and PCI DSS require evidence-based reporting. An AI-generated compliance report is worthless to an auditor unless every finding traces to verifiable source data. With HMAC-SHA256 signing, every finding in a compliance report links back to a cryptographically verified collection event. When an auditor asks “how do you know this firewall rule exists?” the answer is a specific SSH session with a valid cryptographic signature — not “the AI said so.”
The Difference Between AI-Assisted and AI-Trusted
“AI-assisted” means an operator uses AI as one input among many and always verifies independently. “AI-trusted” means the operator can rely on AI output to make operational decisions without re-checking every data point by hand. Cryptographic verification is what makes that transition possible. The operator does not trust the AI — they trust the math. The HMAC either verifies or it does not.
Without cryptographic anti-fabrication, AI in infrastructure is a research toy — interesting but not operational. With it, the platform becomes a tool that operators can actually depend on at 3 AM when a circuit goes down and they need to understand the blast radius in sixty seconds.
The bottom line: Every AI operations platform will tell you their AI is accurate. This platform is the one where accuracy is mathematically enforced. The AI cannot hallucinate a router config because any output it generates will lack a valid HMAC-SHA256 signature and be automatically flagged. This is not a marketing claim. It is a cryptographic property enforced by a compiled binary that the AI cannot access, running in a separate process that the AI cannot read, using a key that exists only in volatile memory. Math, not promises.