Agent-Native Dataset Design
Published Apr, 25 2026. Preprint on Zenodo; evaluation code is private, available on request.

Field note - version 1.10.0
Across 106 recorded runs of 9 AI agent seats operating Orbyt Labs between Jul, 21 2026 and Sep, 26 2026, 91 passed the workflow's completion gate (86%), 15 were refused by it, and 73 files an agent changed outside its lane were erased before commit; the frozen Q3 2026 baseline was 80.0% on 30 runs.
Collected Sep, 11 2026 to Sep, 26 2026. Sample: 106 recorded runs of 9 seats in one organization.
| Metric | Value | How it is counted |
|---|---|---|
| Recorded runs | 106 | public/autonomy-ledger.json, current.perSeat, sum of runs |
| Seats with a run ledger | 9 | same file, perSeat length |
| Runs whose output passed the completion gate | 91 | same file, sum of completions |
| Completion rate | 86% | completions / runs |
| Runs the guards refused | 15 | same file, sum of failures |
| Rows where no run happened (skip, halted, infrastructure) | 4 | same file, excluded from both sides of the rate |
| Files an agent changed outside its lane, erased before commit | 73 | same file, sum of discards |
| First recorded run | 07-21-2026 | same file, earliest firstRun |
| Last recorded run | 09-26-2026 | same file, latest lastRun |
| Frozen Q3 2026 baseline completion rate | 80.0% | same file, baseline.totals (frozen 08-11-2026, never regenerated) |
| Runs in the frozen baseline | 30 | same file, baseline.totals.runs |
Every figure is read from the organization's committed per-seat run ledger (docs/org/stats/<seat>.jsonl, aggregated into public/autonomy-ledger.json by generate-stats.cjs at every commit). A run is one scheduled or dispatched execution of a seat's workflow; the ledger row is written by the workflow itself, not by the agent.
Completion means the run's output survived the workflow's completion gate (a check of how the agent's run ended: its exit, its result and its turns against its cap), the guard gauntlet, and the commit. Since decision D-0061, in September 2026, the completion line a seat prints is recorded as its claim rather than required; runs before that decision also had to print it. A run whose output the guards refused is a failure. Rows recording that no run happened (skip: nothing to do; halted: a kill switch; infrastructure: the org could not run) are counted separately and sit on neither side of the rate.
A discard is a file the agent changed outside its effective keep set, erased before the commit rather than shipped. It is an aggregate counter across runs and a different unit from a failure: it says the agent reached outside its lane, not that its work was rejected.
The baseline is the ledger as first published on Aug, 11 2026 (30 runs, 80.0% completion), frozen on purpose and never regenerated, so every later reading is measured against the same origin rather than against a moving average.
Failures are printed, never netted out. The rate is completions divided by runs, and both numbers are on the page.
This URL is permanent. When the data is refreshed the version bumps and a row lands here, so a citation made today still resolves to the finding it cited.
| Version | Date | Change |
|---|---|---|
| 1.10.0 | 09-26-2026 | Refresh: runs 105 -> 106; completions 90 -> 91; discards 72 -> 73; lastRun 09-25-2026 -> 09-26-2026. |
| 1.9.0 | 09-25-2026 | Refresh: runs 103 -> 105; completions 88 -> 90; completionRate 85% -> 86%; discards 71 -> 72; lastRun 09-24-2026 -> 09-25-2026. |
| 1.8.0 | 09-24-2026 | Refresh: runs 102 -> 103; completions 87 -> 88; lastRun 09-23-2026 -> 09-24-2026. |
| 1.7.0 | 09-23-2026 | Refresh: runs 104 -> 102; completions 86 -> 87; completionRate 83% -> 85%; failures 18 -> 15; notRun 1 -> 4. |
| 1.6.0 | 09-23-2026 | Refresh: runs 102 -> 104; completions 84 -> 86; completionRate 82% -> 83%; discards 69 -> 71; lastRun 09-22-2026 -> 09-23-2026. |
| 1.5.0 | 09-22-2026 | Refresh: runs 101 -> 102; completions 83 -> 84; discards 63 -> 69. |
| 1.4.0 | 09-22-2026 | Refresh: runs 91 -> 101; completions 76 -> 83; completionRate 84% -> 82%; failures 15 -> 18; discards 59 -> 63; lastRun 09-17-2026 -> 09-22-2026. |
| 1.3.0 | 09-17-2026 | Refresh: runs 83 -> 91; completions 69 -> 76; completionRate 83% -> 84%; failures 14 -> 15; discards 54 -> 59; lastRun 09-12-2026 -> 09-17-2026. |
| 1.2.0 | 09-12-2026 | Refresh: runs 82 -> 83; completionRate 84% -> 83%; failures 13 -> 14; discards 52 -> 54; lastRun 09-11-2026 -> 09-12-2026. |
| 1.1.0 | 09-12-2026 | Refresh: notRun 0 -> 1. |
| 1.0.0 | 09-11-2026 | First publication. |
The other measurements from the same repository, and the two papers they sit beside.
Published Apr, 25 2026. Preprint on Zenodo; evaluation code is private, available on request.
Preprint published Sep, 2 2026 on this site. Zenodo deposit landed Sep, 9 2026, DOI 10.5281/zenodo.22683647.
Point-in-time census of the repository, Sep, 25 2026.
Bartak, J. (2026). What an Agent Seat Actually Completes. Orbyt Labs Research, version 1.10.0. https://www.orbytlabs.ai/research/agent-seat-completion
CC BY 4.0. Reuse it with attribution.