A rule that cannot fire looks like a rule that passes

Start with what was working

This is not a post about guardrails failing, and I want to establish that before the two cases, because the interesting part depends on it.

The setup is the one described in the multiplier post: an always-loaded instructions file, a decision graph in ADRs, memory files, and an automation layer of hooks and skills that turns the written rules into things a machine refuses. On the production platform that layer is 37 hooks and 38 skills. On a Rust side project I started later it's 11 hooks and 37 numbered rules. Both have been running for months.

By the ordinary measures it works. The import graph across eight bounded contexts carries exactly one accepted cross-module violation, captured in a baseline, with anything new failing the build. Across 2,465 production TypeScript files there are 94 occurrences of : any, eleven as any, and two @ts-ignore. The Rust workspace is clippy-clean with unsafe_code = "forbid" and both findings from an external security review closed with the regression test named in the source above each fix.

The sharpest single datum: earlier this month I ran an adversarial audit across both codebases, roughly 580,000 lines. The most serious thing it found in the application code was an endpoint with its auth guard commented out — and my own findings ledger had already filed it, with an executable probe, three weeks earlier. The system found it first. The audit's contribution was noticing it was still open.

So the enforcement was doing its job. The gap was somewhere else, and it took me a while to see what kind of gap it was.

The question the system could not answer about itself

Every rule I write produces one of two observable outcomes: it blocks a commit, or it says nothing. The second outcome is overwhelmingly the common one, because most commits are fine. That's the design working.

But it means silence is ambiguous, and the ambiguity is total:

Everything else in the system has an answer to this. Tests fail when the code is wrong. Types fail when the shape is wrong. The linter fails when the syntax is wrong. But a governance rule's null result is its normal result, so a broken rule degrades into permanent silence and there is no signal anywhere. It doesn't fail. It stops working.

I found two of these. Neither caused an incident. Both are, I think, the ordinary condition of most repositories that have hooks.

Case one: eight days, and two fixes that both read as correct

The Rust project has a rule capping source-file size — a file has to stay under what a person can hold in their head. When the client UI gained a TypeScript build, I extended the rule to .ts files. Written 25 August. It could not fire until 2 September.

The first version of the call sat below a filter:

case "$file" in
  *.rs) ;;
  *) continue ;;   # every .ts file exits here
esac
...
check_file_size "$file"

Obvious in hindsight. I moved the call above the filter, confirmed the logic by reading it, and shipped the fix with a comment saying the bug was fixed. It still could not fire, because the caller was handing the loop a file list git had already narrowed:

git diff --cached --name-only -- '*.rs'

The rule was now reachable and correct. There were simply no .ts paths in the argument, and there never had been.

Here is the part worth taking away. I had tested the rule. The test called the rule function with a .ts path and asserted it blocked — and it did, because the function was right both times. The defect was never in the rule. It was in what the rule was being handed, and a test that supplies its own argument cannot see that:

Case two: the same shape, on the platform, for weeks

The production repo had a routing gate that decided which skill should be engaged for a given change. It was keyed to a project directory hardcoded from the machine it was written on. On my other machine, running a different OS, the path never matched.

It enforced nothing for weeks. It also never errored, never warned, and never appeared in any output — because a gate whose precondition is unmet is a gate that returns zero.

Two different mechanisms — a shell filter, a hardcoded path — and one shape: the rule was fine and never reached the thing it was meant to judge. That's the class. Once you're looking for it, the question stops being "is this rule correct?" and becomes "what proves this rule ran?"

Three mechanisms that answer it

1. Test the hook, not the rule

The tests now stand up a scratch git repository, copy the real hooks into it, stage real content, and invoke the actual pre-commit — then assert on what the hook printed. Copied rather than symlinked, so a test can't accidentally depend on the parent repo's contents.

Every case has its negative twin. It isn't enough that a rule blocks the bad file; it has to stay silent on the good one, or you've built something that blocks correct work — which is how people learn to type --no-verify, and a bypassed gate is worse than no gate because it still produces the feeling of coverage.

Twenty-six cases on the platform, eleven on the Rust project. Both suites run green today. The first run of the platform suite immediately caught a false block I'd have shipped: my own documented example of a valid trailer contained a comma inside a parenthetical note, and the parser stripped parentheticals after splitting on commas.

2. If a name is required, the name has to resolve

Commits in both repos carry a trailer naming which skill produced the change. Requiring it and not checking it produces something that looks like an audit trail and is fiction.

When I finally checked: 73 trailers in the Rust project named skills that had never existed. On the platform, roughly 148 instances covering 66 distinct names that resolve to nothing — including one that was simply the right name with an underscore where a hyphen belonged, passing silently for months.

The checker has to accept more than the local skills directory — plugins, built-ins, an explicit "none" — because 231 of the trailers were honest and rejecting them would have made the gate the enemy. That balance is most of the work in any rule like this.

3. A test you never watched fail proves nothing

This one came out of a real security fix. A nonce in the Rust project repeated after 65,536 fragments — about a minute of screen sharing — which under the AEAD in use isn't a degraded stream, it's a broken one. There was a test covering that path. It had never been observed failing.

So a change inside the defined proof set — the places where defects have historically been silent and expensive — has to name the test it was watched failing on, with the failure line:

Tripwired: screen_nonces_never_repeat_across_the_u16_wrap
           — failed on 8d72024d with "assertion failed: nonces.insert(n)"

Or waive it explicitly, in writing, with a reason. The scope matters as much as the rule: a tripwire on every commit would fire on documentation changes and be ignored within a week.

The gate I built, then deleted

Worth including because it's the failure mode these mechanisms are for.

I had a review step emit a receipt into the commit message — a trailer proving the review had run, with CI as the backstop. It was wrong in three ways at once. The hash covered the public diff, so anyone could forge one. The verdict word sat inside an unchecked wildcard, so CI accepted a BLOCK receipt exactly as it accepted a GOOD one. And because the honest flow always produced a receipt, CI's backstop never once fired in anger.

A gate that manufactures confidence without checking anything is strictly worse than no gate, because you stop looking. I deleted it. What replaced it doesn't write anything into the commit at all: the durable artefact is a ledger entry carrying an executable probe, and CI re-runs the probe rather than trusting any claim about the review. A finding that stays open therefore gets louder with age instead of quieter.

The replacement review step has the same shape one level up. It can block, but only if it quotes a line that genuinely exists at that line of the staged blob — so it cannot block without pointing at real code it has read. Everything else about it fails open, loudly. An assertion isn't a gate until something re-checks it.

Why the small repository found what the large one couldn't

All three mechanisms were built in the Rust project first and ported back into the platform afterwards. That direction surprised me, and the reason is structural rather than accidental.

A silent gate on the platform is one of 37, in a tree of half a million lines that nobody holds in mind at once. The same silent gate in the side project is one of 11, in a tree one person wrote in nineteen days. The signal is identical; the noise floor isn't. Governance defects are invisible at 500K lines and audible at 80K, and the only thing that changed between the two was how much else was going on.

The second reason is the one I'd have got wrong in advance. I don't know Rust deeply — TypeScript and NestJS are where I'm fluent. On the platform I can unconsciously compensate for a weak gate, because I know what the code should look like and I notice when it doesn't. In Rust I can't. Whatever the machinery misses, I miss.

Which means the greenfield project was functioning as a test harness for the method itself, and the unfamiliarity was the instrument rather than a handicap. If you want to know whether your process actually carries you, run it somewhere your intuition can't quietly do the work.

What ported, and what that says about portability

Moving the mechanisms from a Rust workspace into a NestJS monolith took a day. That surprised me less once I classified what the rules actually assert. Of the 37 in the Rust project, roughly 32 are architectural and five are language idiom:

• Architectural — transfers anywhere: no I/O in the domain layer; this component must never link that one; tests may not sleep or read the wall clock; a wire-format change bumps the version; enum variants are appended, never inserted; an expensive key derivation sits behind a permit; the documents may not lose races with the code; a file stays under the size a person can hold.
• Language idiom — needs a local equivalent: the unwrap/panic ban, println!, module-file naming. And even these are local expressions of universal shapes — "library code models absence rather than aborting", "nothing writes to stdout from a library".

So the machinery is largely language-agnostic and much less repo-agnostic. The rules are portable in shape and specific in content — and the content is specific to a codebase's incident history, not to its syntax. Nearly every rule in either repo cites the event that produced it: a review, an outage, a date. That's the part you can't copy from someone else's repository, and it's the part that makes the whole thing work.

The honest scoreboard

Two rules weren't running. One for eight days, one for weeks. Neither caused an incident, and both were found by the system getting more rigorous rather than by something going wrong.

What I'd take from that isn't that the guardrails were unreliable. Every rule that ran ran correctly, for months, across two languages, and caught things I would not have caught. The gap was in knowing, not in doing: I could tell you what the rules said and I could not tell you which ones were still executing. Those are different properties, and only the first one is what most people mean when they say a codebase has quality gates.

The question is cheap to ask and slightly uncomfortable to answer, so here it is for your own repository: pick a hook you rely on, break the thing it's supposed to catch, and commit. If it stays quiet, you've learned something. If you don't know how you'd even test that, that's the same answer in a different form.

I'd been running this system for months and writing about it before I thought to ask.