84 Ways to Tell Me I'm Wrong.

No human reads 532,000 lines of code.

That is what my eight projects total right now, covered by 21,454 tests, built and operated by one person. I ship into it every day. So do twelve agent seats. The reading cannot be delegated to attention, because attention does not scale and mine is already spent. It has to be delegated to something deterministic.

That something is an audit harness. It runs 84 numbered dimensions in about six minutes, and its live scoreboard is public on Orbyt's building-in-public page.

This post is the README in narrative form. The point is not that I have one. The point is how each dimension was earned, because that method transfers and my dimension list does not.

Why is test coverage not enough?

Because tests prove the behavior you thought to specify. Every failure class below shipped past a green suite.

Start with the worst one. A test whose expected value comes from the same source as its actual value can never fail. In one of my routes, the log table recorded a 400 while the customer received a 404. Contract tests asserted the response and were correct. Wrapper tests asserted the log row and recorded whatever the code passed, which is a tautology. Nothing asserted that the two agreed.

13,364 passing tests missed it for a month. It was found by querying the production table: 1,564 rows saying 400 that were really 404, on the same rows a paying customer reads.

Then there is the mock that copies the bug. A model default moved underneath me and the response shape changed, so the first content block became a thinking block with empty text. Nine call sites read content[0].text. A scheduled job failed silently for a month. The root enabler was that not one mock in the entire suite included the type discriminator the real API always sends. 14,271 passing tests could not see it, because every one of them was a copy of the wrong assumption.

Tests are also structurally blind to whole categories. Whether every route has a latency budget. Whether the log and the response agree. Whether a cron reporting eighteen consecutive successes is actually at 98.4 percent of its hard timeout and counting down.

None of that is a bug in a function. All of it ships.

What the three tiers actually are

The harness is not one pass. It is three, and the split is what makes it survivable.

Dimension 36 carries the lowest number and runs last. It auto-renumbers to the highest dimension on the public scoreboard so it stays at the end no matter how many get added. LLM-judged grades are not allowed to block until they prove run-over-run stability on unchanged anchors.

That restraint matters more than the checks. A grade that moves when nothing changed is not a measurement.

Six minutes, in three rounds

The ordering is not cosmetic. Run these chronologically, in the order the dimensions were numbered, and the build blocks every cheap check behind it.

Round 3 has to serialize, and the reason is instructive. Those checks share a per-IP rate window against the app's own limiter. Run them in parallel and they fail each other with 429s, and every one of them is correct alone and broken in company. A check that talks to a rate-limited surface must either own the whole window or absorb exactly one window of contention. Anything else is a coin flip wearing a pass/fail badge.

The fail-fast rule falls out of the ordering. Any Round 1 failure stops the run. A typo caught by a locale check in 140 milliseconds should never wait behind a three-minute build.

How each dimension was chosen

One rule governs the whole list.

A dimension has to be earned by a failure that already happened. Never invented from a best-practices article.

The failure corpus holds 51 recorded incidents. 29 of them are now a numbered dimension. The gap between those numbers is the honest part: half the things I have learned are still only written down, and writing something down does not enforce it.

I know exactly what that costs. I wrote a lesson saying to alert on duration against the ceiling rather than on the last status. Eight days later the identical shape recurred in a different subsystem, because the lesson was prose and prose does not run.

A lesson is a hope. A guard is a fact.

The obvious version of this harness is a universal debt scanner: count TODOs, long functions, any, unused exports. I built that version and rejected it. It generates infinite findings nobody acts on, and a report nobody reads is worse than no report, because it teaches you to skim past the real finding sitting next to it.

The dimension that replaced it adds no detection at all. Not one new rule or pattern. It only asks whether the debt already being tracked is going up or down. The most valuable check I have written detects nothing.

Two more design rules earned the same way.

Every escape hatch requires a written reason. A file can opt out of any sub-check with a marker comment, and the marker takes a reason string. Not a flag, a sentence. That single constraint is what keeps the allowlist from becoming the place findings go to die, because writing "skipped because it was annoying" is something people will not do in a diff.

Scoring is honest about partial credit. A warning counts as half, not as a pass. The grade is passes plus half the warnings over the total. That surfaces genuinely suspect sites for a human to look at without either drowning me in known-good patterns or letting a warning quietly count as clean.

And when a dimension turns out to be a bad idea, the number is retired rather than reused. I evaluated a visual regression dimension and declined it: on a fast-iterating site every intended redesign fails the baseline, which invites reflexively updating the snapshots, which defeats the check. Its number is a gap in the list on purpose, because a reused number would make old commit messages lie.

Three dimensions, walked all the way through

Dimension 82: does every registered dimension actually speak?

The dimension that audits the audit. It exists because of what a sweep found on 2026-08-05.

All 62 registered dimension scripts were run and sorted by what each one actually did. No crashes. No timeouts. And three that produced zero bytes and exited 0.

Two of them were the security and coverage scripts. Both are dual-mode: bare, they read the staged diff, which is their pre-commit job. On a clean checkout that is zero files, so they printed nothing and exited successfully. Every audit that had run them bare recorded a pass.

The scorecard said "Security: all checks passed" from a run that opened no file.

Both now declare NOT OBSERVED instead of exiting silently, and audit mode states its own coverage: "no findings across N code files scanned." The fix is one sentence and the lesson is the whole harness: a check that cannot distinguish clean from unexamined is decoration.

Dimension 71: production truth

Every other dimension reads the repository. That means every other dimension inherits the same blind spots as the code, because an assertion whose expected value comes from the same source as its actual value can never fail.

Dimension 71 reads live state instead. It is the loop-breaker.

It is also where I learned to distrust authoritative-looking numbers. A table listing reported role_occupation_codes at 0 rows when it held 60, and the migration ledger at 0 when it held 84. Those are Postgres planner estimates, stale between analyze runs. Two findings were reported off them and both had to be retracted.

Any claim about how much data exists now comes from an exact count, never from a listing.

Dimension 37: the selection rule that optimized for its proxy

This is the one I would show a skeptic, because nothing in the repository could have caught it.

The kept-set for programmatic salary pages seeded 40 curated roles, then filled the remaining 110 of 150 slots by sorting the catalog on median pay. On a 3,445-row catalog containing every synthetic executive micro-title, ranking by pay is ranking by seniority.

All 110 slots went to titles like senior vice president of security and roughly 100 variations of vice president tied at the same salary, each carrying 50 city pages. Roughly 5,500 pre-built pages about what a vice president of partner engineering earns in Miami.

Meanwhile software-engineer in San Francisco returned 404. So did registered nurse in New York, dentist in Phoenix, accountant in Dallas, and 14 more combinations taken from real recorded user queries. Eighteen out of eighteen.

Nothing was internally inconsistent, which is exactly why it survived eight weeks and a full green suite. The page generator and the sitemap both import the same kept-set, so the built pages and the advertised URLs agreed perfectly with each other and with the code. Every guard in the repository reads the repository.

The gap was only visible from outside, through search console query data: of 61 roles named in real queries, 50 had no page at all.

The doc comment directly above the curated list already said seniority variants carry no search demand and were deliberately excluded. The very next function put 110 of them back.

A doc comment stating an intent that the adjacent code contradicts is a bug report nobody filed.

One detail I keep, because it is the honest limit of the fix: a page that 404s cannot earn an impression, so demand data gathered through the gap is censored. It reads as a floor on the loss, never an estimate of it. That is why the fix is a curated list rather than a list derived from observed data, which would have locked in the blind spot.

What did it catch that the tests did not?

Every row has the same shape. A correct-looking artifact and a green suite.

What it still misses

This is the section most tooling posts skip, so it is the one worth reading.

It cannot judge taste. Whether a feature should exist, whether the copy is any good, whether the abstraction is the right one. Dimension 36 gestures at this with an LLM judge and I do not let it block, because I have not proven its grades are stable enough to trust.

It cannot see a failure class nobody has hit. Every dimension is a scar. The list is a map of my history, not a map of the risk.

It cannot see anything that only exists in production unless an instrument is pointed at it. The accessibility dimension runs axe-core against a live server for exactly that reason, and it is one of four checks that need a running app rather than a file tree. Dimensions 71 and 76 read live state and they are the exception, not the rule. The cron burning toward its ceiling was found by accident, during a sweep asking a different question.

And it cannot save you from a detector that is wrong. One of mine false-fired critical for 16 days, and the session reading it concluded the data was broken rather than the check. A red the harness cannot justify corrodes trust exactly the way a false green does.

How to adapt this

Do not copy my 84 dimensions. They are my scars.

Open your incident channel and read the last twenty postmortems. For each one, ask a single question: what deterministic check, running in under a second, would have caught this before it shipped? Most will have an answer. That answer is dimension one.

Then hold four rules.

Prove every detector by injecting the defect and watching it go red. A guard you have only ever seen pass is decoration. "It passes" and "it can fail" are different facts, and only one of them is evidence.

Ship the negative case with it. The check must stay quiet on the healthy tree. My first draft of one guard reported 23 violations that were all false positives. Suspect the detector first.

Separate blocking from reporting. Anything judged rather than measured reports until it earns the right to block. Anything that would flood you reports. A permanent false positive in a report-only check is worse than no check.

Fail fast and cheap first. Twenty sub-second static checks run before anything expensive. A typo caught in 140 milliseconds should never wait behind a three-minute build.

What a CTO can do Monday

Count the lines in your largest repository. Divide by the number of engineers who have read any of it in the last quarter. That ratio is your real coverage number and it is worse than your test coverage number.

Then pick one incident from this year and mechanize it. Not a document, not a checklist item, not a bullet in an onboarding deck. A script that exits non-zero.

Everything else on this list is scale. That first one is the decision.

A codebase nobody can read is not a liability because it is large. It is a liability because nothing is reading it.

Related reading:

• I Can't Read My Own Codebase. I Ship Daily. the argument this harness is the tool for
• Why Build an AI Test Harness the layer underneath the audit, and why tests come first
• You Ship What You Tolerate the standard a harness exists to enforce when attention runs out
• Verification Is the New Literacy why checking the output is now the load-bearing skill
• Self-Healing Is a Euphemism. the repairs that run before a guard ever fires, and why you count them

---

Originally published on justinbartak.ai on Aug 23, 2026.