Field note - version 2.4.0
Every Guard Must Prove It Can Stay Silent
As of 17 September 2026, the Orbyt Labs repository enforces 114 automated guard scripts, of which 16 block a commit outright, and 99 of 99 (100%) guard test suites assert that the detector stays silent on a clean input rather than only firing on a bad one.
Collected 08-27-2026 to 09-17-2026. Sample: 114 guard scripts and 99 guard test suites in one repository.
The data.
- Block a commit outright16 of 114
- Carry a STRICT promotion switch89 of 114
- Test suites proving the guard stays silent99 of 99
| Metric | Value | How it is counted |
|---|---|---|
| Guard scripts in the repository | 114 | ls scripts/check-*.cjs | wc -l |
| Blocking a commit outright | 16 | node "$ROOT/scripts/check-<name>.cjs" || exit 1 lines in scripts/pre-commit |
| Carrying a STRICT promotion switch | 89 | grep -l _STRICT scripts/check-*.cjs | wc -l |
| Guard test suites | 99 | ls __tests__/scripts/check-*.test.ts | wc -l |
| Test suites proving the guard stays SILENT | 99 of 99 (100%) | node scripts/check-doc-claim-parity.cjs, check 6 |
| Audit dimensions in the roster | 97 | app/(site)/founder/building-in-public/audit-dims.json, field `total` |
| Logged failures with a guard built in response | 54 | public/failure-corpus.json, field `mechanized_count` |
How it was measured.
A census of one repository on one date, taken by running each command in the source column and reading its output, not by reading documentation that describes the repository.
A guard is any executable at scripts/check-*.cjs. Blocking means the pre-commit hook runs it as `node "$ROOT/scripts/check-<name>.cjs" || exit 1`; the hook invokes no guard through an npm script, so that one form is the whole population.
A STRICT switch means the guard reads a <NAME>_STRICT environment variable, which promotes it from reporting to blocking. Most guards are report-only by default and can be promoted per run.
The stays-silent figure uses the repository's own detector rather than a definition written for this entry. Check 6 of check-doc-claim-parity.cjs scans every __tests__/scripts/check-*.test.ts for absence idioms (.toBe(0), .toBe(false), .not.to, not.toMatch, not.toContain, toEqual([]), toHaveLength(0), toEqual({})) and warns on any file with none.
The rule being measured exists because four detectors misfired on a single day in July 2026, every one of which passed its own test suite, because each suite only ever proved the detector could be LOUD.
What this does not show.
- One repository, one team, one date. There is no control group and no comparison against other codebases, so this describes a practice rather than demonstrating that the practice works better than an alternative.
- A census, refreshed on a schedule, not a controlled time series. The changelog carries every refresh with its deltas, but a rising count can mean growth or reclassification, and no trend is claimed from it.
- An absence idiom in a test file is evidence that a negative case was written, not proof that the negative case is a good one. A test can assert zero findings against an input that could never have produced any.
- The stays-silent detector is deliberately conservative and its author documents two earlier drafts that were wrong in opposite directions, one matching loose prose and one omitting toEqual([]). It is a warning rather than a blocking gate for that reason.
- Counting guards is not measuring defects caught. This entry does not claim the 114 guards prevented 114 problems.
Revisions.
This URL is permanent. When the data is refreshed the version bumps and a row lands here, so a citation made today still resolves to the finding it cited.
| Version | Date | Change |
|---|---|---|
| 2.4.0 | 09-17-2026 | Refresh: guards 113 -> 114; strict 88 -> 89; suites 98 -> 99; silentSuites 98 of 98 (100%) -> 99 of 99 (100%); auditDims 96 -> 97; guardedLessons 51 -> 54. |
| 2.3.0 | 09-13-2026 | Refresh: guardedLessons 50 -> 51. |
| 2.2.0 | 09-13-2026 | Refresh: guardedLessons 49 -> 50. |
| 2.1.0 | 09-12-2026 | Refresh: guardedLessons 48 -> 49. |
| 2.0.0 | 09-11-2026 | Method change and refresh: guards 105 -> 113; blocking 24 -> 16; strict 80 -> 88; suites 87 -> 98; silentSuites 87 of 87 (100%) -> 98 of 98 (100%); auditDims 90 -> 96; guardedLessons 59 -> 48. Every row is now produced by scripts/refresh-research-notes.ts. Two 1.0.0 figures could not be reproduced by their own stated commands on the 2026-08-27 tree: blocking was recorded as 24 where the hook's own form counts 16, and the guard-built-in-response row (59) counted every logged failure; it now reads the failure corpus's mechanized_count, which is checked against the tree. |
| 1.0.0 | 08-27-2026 | First publication. Initial census. |
More from Research.
The other measurements from the same repository, and the two papers they sit beside.
Cite this.
Bartak, J. (2026). Every Guard Must Prove It Can Stay Silent. Orbyt Labs Research, version 2.4.0. https://www.orbytlabs.ai/research/guard-harness
CC BY 4.0. Reuse it with attribution.