Agent-Native Dataset Design
Published Apr, 25 2026. Preprint on Zenodo; evaluation code is private, available on request.

Field note. Version 2.17.0
As of Sep, 25 2026, the Orbyt Labs repository enforces 118 automated guard scripts, of which 16 block a commit outright, and 104 of 104 (100%) guard test suites assert that the detector stays silent on a clean input rather than only firing on a bad one.
Collected Aug, 27 2026 to Sep, 25 2026. Sample: 118 guard scripts and 104 guard test suites in one repository.
| Metric | Value | How it is counted |
|---|---|---|
| Guard scripts in the repository | 118 | ls scripts/check-*.cjs | wc -l |
| Blocking a commit outright | 16 | node "$ROOT/scripts/check-<name>.cjs" || exit 1 lines in scripts/pre-commit |
| Carrying a STRICT promotion switch | 93 | grep -l _STRICT scripts/check-*.cjs | wc -l |
| Guard test suites | 104 | ls __tests__/scripts/check-*.test.ts | wc -l |
| Test suites proving the guard stays SILENT | 104 of 104 (100%) | node scripts/check-doc-claim-parity.cjs, check 6 |
| Audit dimensions in the roster | 100 | app/(site)/founder/building-in-public/audit-dims.json, field `total` |
| Logged failures with a guard built in response | 67 | public/failure-corpus.json, field `mechanized_count` |
A census of one repository on one date, taken by running each command in the source column and reading its output, not by reading documentation that describes the repository.
A guard is any executable at scripts/check-*.cjs. Blocking means the pre-commit hook runs it as `node "$ROOT/scripts/check-<name>.cjs" || exit 1`; the hook invokes no guard through an npm script, so that one form is the whole population.
A STRICT switch means the guard reads a <NAME>_STRICT environment variable, which promotes it from reporting to blocking. Most guards are report-only by default and can be promoted per run.
The stays-silent figure uses the repository's own detector rather than a definition written for this entry. Check 6 of check-doc-claim-parity.cjs scans every __tests__/scripts/check-*.test.ts for absence idioms (.toBe(0), .toBe(false), .not.to, not.toMatch, not.toContain, toEqual([]), toHaveLength(0), toEqual({})) and warns on any file with none.
The rule being measured exists because four detectors misfired on a single day in July 2026, every one of which passed its own test suite, because each suite only ever proved the detector could be LOUD.
This URL is permanent. This table records version numbers and changes.
| Version | Date | Change |
|---|---|---|
| 2.17.0 | 09-25-2026 | Refresh: guardedLessons 66 -> 67. |
| 2.16.0 | 09-24-2026 | Refresh: auditDims 97 -> 100. |
| 2.15.0 | 09-23-2026 | Refresh: guardedLessons 65 -> 66. |
| 2.14.0 | 09-23-2026 | Refresh: guardedLessons 64 -> 65. |
| 2.13.0 | 09-23-2026 | Refresh: guards 117 -> 118; strict 92 -> 93; suites 103 -> 104; silentSuites 103 of 103 (100%) -> 104 of 104 (100%). |
| 2.12.0 | 09-23-2026 | Refresh: guards 116 -> 117; strict 91 -> 92; suites 102 -> 103; silentSuites 102 of 102 (100%) -> 103 of 103 (100%). |
| 2.11.0 | 09-23-2026 | Refresh: guardedLessons 63 -> 64. |
| 2.10.0 | 09-23-2026 | Refresh: guardedLessons 62 -> 63. |
| 2.9.0 | 09-23-2026 | Refresh: guards 115 -> 116; strict 90 -> 91; suites 101 -> 102; silentSuites 101 of 101 (100%) -> 102 of 102 (100%). |
| 2.8.0 | 09-23-2026 | Refresh: guards 114 -> 115; strict 89 -> 90; suites 100 -> 101; silentSuites 100 of 100 (100%) -> 101 of 101 (100%). |
| 2.7.0 | 09-23-2026 | Refresh: guardedLessons 61 -> 62. |
| 2.6.0 | 09-22-2026 | Refresh: suites 99 -> 100; silentSuites 99 of 99 (100%) -> 100 of 100 (100%). |
| 2.5.0 | 09-22-2026 | Refresh: guardedLessons 54 -> 61. |
| 2.4.0 | 09-17-2026 | Refresh: guards 113 -> 114; strict 88 -> 89; suites 98 -> 99; silentSuites 98 of 98 (100%) -> 99 of 99 (100%); auditDims 96 -> 97; guardedLessons 51 -> 54. |
| 2.3.0 | 09-13-2026 | Refresh: guardedLessons 50 -> 51. |
| 2.2.0 | 09-13-2026 | Refresh: guardedLessons 49 -> 50. |
| 2.1.0 | 09-12-2026 | Refresh: guardedLessons 48 -> 49. |
| 2.0.0 | 09-11-2026 | Method change and refresh: guards 105 -> 113; blocking 24 -> 16; strict 80 -> 88; suites 87 -> 98; silentSuites 87 of 87 (100%) -> 98 of 98 (100%); auditDims 90 -> 96; guardedLessons 59 -> 48. Every row is now produced by scripts/refresh-research-notes.ts. Two 1.0.0 figures could not be reproduced by their own stated commands on the 08-27-2026 tree: blocking was recorded as 24 where the hook's own form counts 16, and the guard-built-in-response row (59) counted every logged failure; it now reads the failure corpus's mechanized_count, which is checked against the tree. |
| 1.0.0 | 08-27-2026 | First publication. Initial census. |
The other measurements from the same repository, and the papers they sit beside.
Published Apr, 25 2026. Preprint on Zenodo; evaluation code is private, available on request.
Preprint published Sep, 2 2026 on this site. Zenodo deposit landed Sep, 9 2026, DOI 10.5281/zenodo.22683647.
344 sessions and 45,876 turns, measured Sep, 26 2026.
Bartak, J. (2026). Every Guard Must Prove It Can Stay Silent. Orbyt Labs Research, version 2.17.0. https://www.orbytlabs.ai/research/guard-harness
CC BY 4.0. Reuse it with attribution.