Every Guard Must Prove It Can Stay Silent
Point-in-time census of the repository, Sep, 25 2026.
Preprint. CC BY 4.0. Sep, 3 2026
Governing an Agent Leadership Team: Guardrail, Kill-Switch, and Liveness Patterns from a One-Human Autonomous Organization.
Download PDF →Check this PDFView on ZenodoView on GitHubFailure corpus as read (JSON)Study runs, fixed population (JSON)
Use the fixed study file for the paper's run totals.
12
Named agent seats, registry read 09-02-2026
66
Documented failures, corpus read 09-02-2026
65
Seat runs, 07-21-2026 to 09-01-2026
45
Logged decisions, 07-13-2026 to 09-02-2026
1
Global halt, set and lifted 08-20-2026
Abstract
Half of the documented engineering failures at one company running an agent organization, 33 of 66 in a single-rater, agent-coded corpus, were failures of its own instruments: tests, guards, gates or metrics that could not fail, could not see, or measured a proxy. We operate the organization studied: twelve named seats, nine running a language model, hold a reporting, proposing and publishing cadence under one accountable human. The corpus (`public/failure-corpus.json`, Sep, 2 2026) spans Jun, 9 2026 to Sep, 2 2026 with eleven undated items; 42 of 66 carry a named countermeasure, and verification gaps are the largest class at 33.
The ledger (`public/autonomy-ledger.json`, Sep, 2 2026) records 65 seat runs over six weeks, Jul, 21 2026 to Sep, 1 2026, at 84.6 percent completion, six of the 55 completions being panel-written observer rows; the frozen 80.0 percent baseline shares its 30 rows, and the 4.6-point gap is within two runs' movement. Forty-five decisions went to an append-only log (`docs/org/decision-log.md`); the organization kill switch was pulled once, on Aug, 20 2026. The corpus has no agent-failure class and one rater assigned every class, so the share is exploratory rather than a comparison.
The design law that survived is positive observation: a control is live only when its refusal has been observed, and a monitor that could not look reports NOT OBSERVED rather than clean. That verdict is printed; at the layer that escalates, a monitor that could not look and one that saw health are the same event. We contribute a three-tier guardrail model, positive-observation liveness, the published corpus, ten design principles, and a retrofit checklist.
Key findings
Five results from the record, ranked by load-bearing weight on the paper’s thesis. Every number carries the date it was measured.
1. Half of the documented failures were failures of the instruments.
Of 66 failures in the corpus read on Sep, 2 2026, 33 are verification gaps: tests, guards, gates or metrics that could not fail, could not see, or measured a proxy. The corpus carries no class for agent failure; the agents' own failures sit in the run ledger, on a different denominator.
2. Verification gaps are the largest class, and 42 of 66 carry a named countermeasure.
The three-class taxonomy splits the corpus into 33 verification gaps, 18 operator-process failures and 15 external-behavior failures. 42 items carry a named countermeasure (13 guard scripts, 22 tests, one workflow and six other files) that now stands against the failure, which is 63.6 percent (public/failure-corpus.json, measured Sep, 2 2026).
3. The seats completed 84.6 percent of 65 runs against a frozen 80.0 percent baseline.
The fixed study population contains 65 runs from Jul, 21 2026 through Sep, 1 2026. Of these, 55 completed and 10 failed or were rejected. The rows record 47 discards. The completion rate is 84.6 percent, rounded from 55 divided by 65. The live aggregate includes later runs and the baseline frozen on Aug, 11 2026.
4. The organization kill switch was pulled exactly once, and 45 decisions went to an append-only log.
The kill switch is a file. Its whole history is five commits, with one global halt set and lifted on Aug, 20 2026 during a domain change. The decision log held 45 entries on Sep, 2 2026, D-0001 on Jul, 13 2026 to D-0045 on Sep, 2 2026, and permits no edits: a correction is a new entry.
5. The design law that survived is positive observation.
A control counts as live only when its refusal has been observed, and a monitor that could not look reports NOT OBSERVED rather than clean. The fence fails unless it watches a real denial, a canary watches the fence, and the dead-man switch judges run conclusions rather than the ledger the watched system writes.
Reproducibility package
The paper reads the failure corpus, autonomy ledger and decision ledger, generated from the organization's records and published under CC BY 4.0. Readers can recount these original files and the frozen study population without access to the private repository. The failure corpus is published twice: the file exactly as the paper read it on Sep, 2 2026, and the current file, which regenerates as lessons land and has grown since. Re-count the paper against the frozen one.
Failure corpus, as the paper read it
public/research/agent-leadership-governance/failure-corpus-2026-09-02.jsonThe 66 documented failures the paper measured, frozen on Sep, 2 2026: the three-class taxonomy and the 42 named countermeasures that existed in the tree that day. Byte for byte the file at commit 465bd6ff0. CC BY 4.0.
Independent coding, second rater
public/research/agent-leadership-governance/independent-coding-2026-09-13.jsonA different model coded the same 66 items blind on Sep, 13 2026. The file holds the instructions verbatim, the rater's reason for every item, the confusion matrix, Cohen's kappa with its formula and both terms, and every disagreement. Rated by gpt-6-astra through the OpenAI Codex CLI. CC BY 4.0.
Failure corpus, current
public/failure-corpus.jsonThe same record as it stands today, regenerated whenever a lesson lands. It has grown since the paper read it, so its count is not the paper's count; cite the frozen file for the paper's numbers. CC BY 4.0.
Study runs, fixed population
public/research/agent-leadership-governance/study-runs-2026-09-15.jsonCount rows by outcome and sum discarded_count to reproduce the run totals. The fixed file includes its selection rule and SHA256. CC BY 4.0.
Autonomy ledger, live series
public/autonomy-ledger.jsonCounts by seat, updated from committed run records. Includes the frozen baseline and metric definition.
Decision ledger
public/decision-ledger.jsonThe public projection of the append-only decision log, ids and dates parsed mechanically, prose hand reviewed.
Preprint PDF (self-hosted)
/research/agent-leadership-governance/agent-leadership-governance.pdf
The full paper with its appendices. CC BY 4.0. Also mirrored on Zenodo.
Live autonomy ledger page
/orbyt-collective/autonomy-ledger
The same ledger rendered as markup, with the completion rate and its frozen baseline.
Author ORCID
0009-0005-2615-3624
Justin Bartak (Purecraft LLC). Linked Wikidata entity Q139551829.
Independent coding
The paper’s headline share rests on one rater. The corpus was coded by the agent that operates the organization, and the abstract says so. On Sep, 13 2026 a second rater, the model gpt-6-astra run through the OpenAI Codex CLI, coded the same frozen 66-item corpus blind. It saw each item’s id and public title and nothing else. It did not see the published class, the date, the countermeasure, or the internal forensics.
The two codings agree on 52 of 66 items, 78.8 percent. Cohen’s kappa is 0.67. Kappa discounts the agreement two raters would reach by chance from their class totals. The file below shows the formula with both terms. The second rater places 29 of 66 items in the verification-gap class. The paper places 33 of 66 there.
What this establishes is narrow. One independent rater, blind to the labels, coded the frozen corpus. Each of its 14 disagreements is published below with the rater’s reason. It is not peer review. It is not a human rater. It says nothing about failures the corpus never logged.
Every disagreement
| Id | Title | Paper | Independent | Rater's reason |
|---|---|---|---|---|
| 2 | Realtime sync can echo a write back into itself unless transient UI state stays out of the database | external-behavior | operator-process | The root is persisting transient UI state in the database, which lets realtime synchronization feed a write back into itself. |
| 21 | A convention that has never been executed is not a working convention | verification-gap | operator-process | The title identifies reliance on a convention that has never been carried out, making the failure one of execution discipline. |
| 26 | A tool that runs quarterly has no gate between quarters, and its hand-copied config drifts silently | verification-gap | operator-process | The root is maintaining a rarely executed tool through hand-copied configuration without upkeep between runs, allowing silent drift. |
| 33 | Checks sharing one rate-limit window flake each other; absorb exactly one window or serialize | external-behavior | verification-gap | The checks consume one another's rate-limit allowance, so their failures reflect interference between checks instead of independent evidence about the system. |
| 34 | A fix committed but not pushed does not exist, and a guard grading the working tree will certify it as live | verification-gap | operator-process | The root is leaving the fix unpushed and therefore undelivered, while the working-tree guard conceals that incomplete delivery. |
| 37 | A selection rule optimizes its proxy; verify what exists against measured demand, not against the code | operator-process | verification-gap | The selection rule is judged against its own proxy instead of measured demand, so verification cannot establish whether the resulting selection serves the intended need. |
| 38 | A sync rule that tests equality can never be satisfied by the thing it exists to allow; record provenance instead | operator-process | verification-gap | The sync guard's equality condition excludes the very case it should permit, making its predicate incapable of recognizing valid synchronization. |
| 44 | A sandbox that cannot start reports success, and a fence can apply halfway | verification-gap | external-behavior | The root is sandbox behavior that reports success despite startup failure and can apply confinement only partially, violating assumed execution guarantees. |
| 46 | A guard's exit code is not its verdict; report-only and strict-gated checks exit zero while reporting | operator-process | verification-gap | The root is treating a guard's exit code as its verdict even though the guard can report findings while exiting successfully. |
| 48 | Permissive access policies combine with OR, so one globally true policy defeats every narrow policy beside it, and two individually correct guards can still leave the class unowned | verification-gap | external-behavior | The root is the platform's OR combination of permissive policies, which lets a globally true policy defeat narrower restrictions while gaps between guards leave the exposure undetected. |
| 52 | An upstream service reports a downstream failure in its own vocabulary, so the error names a cause that is not the real one, and four consecutive fixes were aimed at the wrong side of the wire | operator-process | external-behavior | The root is an upstream service translating a downstream failure into misleading error vocabulary, which directs subsequent fixes toward the wrong component. |
| 59 | A field that may contain markup is safe only where it is rendered as markup, and the sink no human ever looks at is the one that keeps shipping the defect | verification-gap | operator-process | The root is failing to maintain the markup field's rendering contract across every sink, with the overlooked sink allowing the defect to persist. |
| 62 | A hand edit to a generated file is one run from silent reversion, and a suite that grades the file instead of a regeneration will never see it | verification-gap | operator-process | The root is editing generated output instead of its source of truth, while a suite that skips regeneration conceals the edit's inevitable reversion. |
| 66 | Calendar month arithmetic overflows into the following month, and a test that computes its expectation the same way agrees with the defect instead of catching it | verification-gap | external-behavior | The root is calendar month arithmetic overflowing beyond the intended month, while a test that repeats the same arithmetic merely conceals that unexpected behavior. |
Cite this paper
Permissive (CC BY 4.0) attribution. Both citations resolve to the same Zenodo record; choose whichever your reference manager expects.
Plain text (APA-style)
Bartak, J. (2026). Governing an Agent Leadership Team: Guardrail, Kill-Switch, and Liveness Patterns from a One-Human Autonomous Organization (Version 1.0). Zenodo. https://doi.org/10.5281/zenodo.22683647
BibTeX
@misc{bartak2026governing,
author = {Bartak, Justin},
title = {Governing an Agent Leadership Team: Guardrail, Kill-Switch,
and Liveness Patterns from a One-Human Autonomous Organization},
year = {2026},
publisher = {Zenodo},
doi = {10.5281/zenodo.22683647},
url = {https://doi.org/10.5281/zenodo.22683647}
}Where this paper lives
The same artifact is mirrored across multiple publication surfaces. Permanent DOI on Zenodo is the canonical citation; this page is the canonical owned URL with full Schema.org structured data.
Zenodo (primary, DOI)
live
ORCID Works
live
Orbyt Research (this page)
live
PDF (self-hosted)
live
Study input (frozen corpus)
live
Independent coding (second rater)
live
Published data files (current)
live
License
Released under Creative Commons Attribution 4.0 International (CC BY 4.0). You may copy, redistribute, remix, transform, and build on the paper for any purpose, including commercially, with attribution.
The three published data files the paper reads are also CC BY 4.0. Quote, redistribute, build on, with attribution to Bartak, J. (2026) for the paper and Orbyt Labs for the data.
Related on this site
Agent-Native Dataset Design
The first paper in the series: the data product this organization operates. DOI 10.5281/zenodo.19754393.
How the Collective works
The machinery end to end, with the ten-term vocabulary the paper uses.
Autonomy ledger
What the seats decide without us, counted per run and never netted out.
Process
How the work actually moves from a seat run to a committed change.
Leadership
The agent officers, their charters, and the one human who holds the line.
Field notes.
The live measurements behind the paper's corpus, refreshed on a schedule: the guard harness, the token bill, the failure log, and the seat run ledger with its denominator.
Read the full paper.
The full preprint covers the machine, the guardrail tiers, the watch chain, the empirical record, ten design principles and a retrofit checklist, with three appendices.