My Paper Overclaimed. So I Built the Claim True.

I've written about putting a fail-closed gate between an AI agent and its tools: every consequential tool call is checked against a plan I approved, and anything outside that plan doesn't run. The design paper behind it made one more promise. Every gate decision, allow and deny, lands in an append-only, tamper-evident log. That log was the evidence. It was what let me say the audit trail came free.

The code kept less of that promise. Every denial was logged. On the allow side, only PowerShell commands were recorded. An allowed file write, the very thing the gate exists to authorize, left no trace at all. Neither did an allowed Bash command or a use of the emergency bypass.

I didn't find this by accident. A verification pass over the repo flagged the paper's logging section as the one place where the public docs claimed more than the shipped code did. It went on the backlog with a plain note: fix it before the claim goes more public.

Two honest fixes

There were two honest ways out. Reword the paper to match the code, or build the code to match the paper.

Rewording was cheaper, and it would have been accurate. It would also have left a real gap. A log that records only exceptions can't tell you the difference between a gate that allowed nothing and a gate that wasn't running. An empty deny log looks exactly like a disabled control. Auditors know this as the difference between exception logging and evidence that the control actually operated. A gate that is meant to produce audit evidence has to record the allows.

Documentation is a claim about the system. If the code doesn't make it true, you're publishing a liability.

So I ruled it a build, not a reword. The paper's new wording would ride in the same pull request as the code, and it would describe exactly what shipped, no more.

The authorizations record

What shipped on 2026-07-15 is one decision log that replaces the three per-surface logs the gate used to keep. Every decision the gate makes on a gated surface (file writes, Bash, PowerShell, and the MCP tools it's configured to gate) appends one JSON row. The row carries the time, allow or deny, the surface, the tool, the target path or the command, a stable decision class, the reason, and the plan that governed it.

Four properties do the real work.

• It's hash-chained. Each row stores the hash of the row before it, and its own hash is a SHA-256 over that link plus the row's canonical JSON. Change or delete a row in the middle and every hash after it stops matching.
• It ships with its verifier. plan-gate.py verify-log re-walks the chain. It exits 0 when the chain is intact, and otherwise it exits 2 and names the first bad row. Anyone can re-prove the record, not just whoever wrote it.
• It protects itself. The gate never authorizes a gated file write into its own log directory, even when an approved plan names that path. The check runs before the always-writable and plan-path rules, so neither can shadow it. A gated agent can't be authorized to touch its own audit trail.
• The bypass is witnessed. The emergency bypass still works, and it still fails open, because a gate that can lock its operator out is its own hazard. But every use now leaves a row.

A row looks like this (values elided):

{"ts": "…", "decision": "deny", "surface": "file", "tool": "Edit",
 "target": "…", "command": "", "class": "log_self_protect",
 "reason": "…", "plan": "…", "cwd": "…", "prev": "…", "h": "…"}

Two older rules carried over unchanged. Logging is fail-open: if the log can't be written, the gate's decision doesn't change. And the log never records file contents or full payloads, only paths, truncated commands and short field prefixes. Concurrent sessions share one log under an advisory lock, and if the last line is ever damaged by a partial write, the chain re-anchors past it and the verifier points at the damaged row.

What "tamper-evident" means here, and what it doesn't

This is the part I most wanted the paper to get right, because overclaiming was how I got here.

The chain makes edits and deletions within the record detectable by anyone who re-runs the verifier. That's the property. It is integrity without a secret: there's no signing key and no external anchor in this version. Someone with ordinary filesystem access to the machine can still cut off the end of the log, or forge a whole new chain from scratch. If that threat matters where you run it, record the latest row hash somewhere else at the end of each session. Then a re-forged chain no longer matches.

There's a second boundary, and it belongs to the gate rather than the log. Self-protection covers the file-write surface. If an approved plan grants a shell command broad enough to write anywhere, that plan could write to the log, and the only control on that is the human reviewing the plan. That blind spot existed before this build. It's documented, not hidden.

The paper's compliance table now states the tamper scope right next to the claim: hash-chained, self-protected, verifiable, and what someone with filesystem access can still do. The shell-command blind spot is written down in the build spec.

Then I asked three reviewers to break it

Before merging, I pointed three independent AI reviewers at the build. Each was read-only and each had a different lens: one checked that the gate's decisions hadn't changed, one attacked the chain, the locking and concurrency, and one went after fail-open behaviour and privacy. Their job was to break it. Every finding came back with a runnable reproduction.

They came back with two blocking findings and two advisory ones.

The .. that got through

The self-protection check compared path strings as written. So a target like <plans>/../logs/plan-gate-decisions.log didn't look like the log directory. It slipped past the check and was allowed, because it started inside a directory the gate was configured to always allow writes to. A gated agent could have been authorized to forge its own audit chain, which is exactly the outcome the feature existed to rule out.

Two of the three reviewers found it independently. The fix checks both the path as written and its fully resolved form, for the target and for the log directory alike. That closes the symlink version of the same attack, and the version where a plan itself lists a .. path.

The verifier that cried wolf

The second blocking finding was subtler. The writer reads a fixed-size window at the end of the log to find the previous row's hash. A file path longer than 64 KiB made a single row bigger than that window, so the next row anchored to the wrong place, and the verifier reported tampering on a log nobody had touched.

That matters as much as a missed attack. A tamper alarm that fires on honest records teaches people to ignore it. The fix caps the length of every field in a row, keeping the worst case far below the window, and widens the window to 256 KiB.

Two quieter ones

A malformed character in the gate's input could stop a row from serializing, so that decision was silently dropped from the record. Every row now writes, with the offending characters replaced. And the writer and the verifier split lines slightly differently, so a damaged line ending in a carriage return produced two different hashes and a false mismatch. Both sides now split the same way.

All four were fixed the same day. Every reproduction is now a regression test, and the plan-gate suite went from 9 tests before this work to 35. The gate-semantics reviewer also ran about 55 differential inputs against the previous version: the only decision that changed was the new self-protection deny, which was the intended one. CI passed and the pull request merged.

The adversarial pass paid for itself on the first finding. The feature that existed to stop an agent forging its audit trail would have shipped with a way to forge the audit trail.

What I'd tell you

If you're relying on an AI agent's logs as evidence, a few things carry most of the weight:

• Log the allows, not just the denials. Exception-only logging can't prove the control ran. An empty log and a dead gate look the same.
• Chain the rows and ship the verifier. A hash chain is cheap to build. The verifier is what turns it from a claim into something anyone can re-check.
• Protect the log from the thing it records, using resolved paths. Compare canonical, resolved paths, never the strings the agent handed you. A .. is all it takes.
• State the tamper scope next to the claim. Say what the chain detects and what it doesn't, in the same paragraph. Leaving that out is how the overclaim started.
• Point adversarial reviewers at your own controls before you merge. Give each one a different lens, ask for a runnable reproduction, and keep every reproduction as a test.

Twenty-one years in regulated industries taught me that documentation is a claim about the system. The paper's claim is true now, and its limits are written down: the tamper scope in the paper, the shell-command blind spot in the spec. The full build spec and the review findings, exploits included, are public in the StrictLock repo, alongside the plan-gate paper. The log sits on the same footing as the externalized memory pattern: state that lives on disk, in git, where it can be checked rather than remembered.

Don't edit the claim down. Build it true, then let someone try to break it.