Agent-Native Dataset Design
Published Apr, 25 2026. Preprint on Zenodo; evaluation code is private, available on request.
Field note. Version 1.0.0
A Single Agent and a Research Team on Equal Ceilings
On Sep, 17 2026, the single agent resolved 1 of 6 briefs and the team resolved 0 of 6. Team retained under the pre registered rule: no.
Measured Sep, 17 2026. Sample: 6 briefs and 12 attempts on the same hypothetical feature.
The data.
- Briefs the single agent resolved1 of 6
- Briefs the team resolved0 of 6
- Briefs the team resolved and the single agent left open0 of 6
| Metric | Value | How it is counted |
|---|---|---|
| Paired briefs | 6 | docs/research/single-agent-vs-team/runs/RESULT.json, briefs.length |
| Assigned attempts | 12 | docs/research/single-agent-vs-team/runs/RESULT.json, trials.length |
| Who performed the substantive reads | agent | docs/research/single-agent-vs-team/runs/RESULT.json, reviewer (absent means founder) |
| Human time clause | not measured: an agent read the reports (Deviations 5) | docs/research/single-agent-vs-team/runs/RESULT.json, human_time_clause |
| Single agent briefs resolved | 1 | docs/research/single-agent-vs-team/runs/RESULT.json, arrangements.single.resolved |
| Single agent accepted findings | 9 | docs/research/single-agent-vs-team/runs/RESULT.json, arrangements.single.accepted_findings |
| Single agent list price cost in USD | 7.84 | docs/research/single-agent-vs-team/runs/RESULT.json, arrangements.single.total_cost_usd |
| Single agent sessions stopped by the ceiling | 0 | docs/research/single-agent-vs-team/runs/RESULT.json, trials.sessions, executed sessions with a budget or cost ceiling stop_clause |
| Team briefs resolved | 0 | docs/research/single-agent-vs-team/runs/RESULT.json, arrangements.team.resolved |
| Team accepted findings | 16 | docs/research/single-agent-vs-team/runs/RESULT.json, arrangements.team.accepted_findings |
| Team list price cost in USD | 16.98 | docs/research/single-agent-vs-team/runs/RESULT.json, arrangements.team.total_cost_usd |
| Team sessions stopped by the ceiling | 0 | docs/research/single-agent-vs-team/runs/RESULT.json, trials.sessions, executed sessions with a budget or cost ceiling stop_clause |
| Briefs the team resolved and the single agent left open | 0 | docs/research/single-agent-vs-team/runs/RESULT.json, briefs.team_resolved_what_single_left_open |
| Team retained under the pre registered rule | no | docs/research/single-agent-vs-team/runs/RESULT.json, team_retained |
| Team preserves every single agent resolution | no | docs/research/single-agent-vs-team/runs/RESULT.json, clauses.preserves_single_resolutions |
| Team resolves a brief the single agent left open | no | docs/research/single-agent-vs-team/runs/RESULT.json, clauses.resolves_additional_brief |
| Team cost does not exceed single agent cost | no | docs/research/single-agent-vs-team/runs/RESULT.json, clauses.cost_not_increased |
| Team human time does not exceed single agent time | no | docs/research/single-agent-vs-team/runs/RESULT.json, clauses.human_minutes_not_increased |
| Registered pool in list price USD | 72.00 | docs/research/single-agent-vs-team/runs/RESULT.json, single session ceiling_usd times trials.length |
| Per attempt ceiling in list price USD | 6.00 | docs/research/single-agent-vs-team/runs/RESULT.json, single session ceiling_usd |
How it was measured.
The unblinded result joins locked verdicts, accepted findings and session costs. The reviewer was the agent. The human time clause was not measured: an agent read the reports (Deviations 5).
The single agent used 7.84 USD at list price with 9 accepted findings. The team used 16.98 USD at list price with 16 accepted findings.
The team resolved 0 briefs the single agent left open. The retention rule also requires preserving every single agent resolution without increasing cost or human time. Finding count does not decide retention.
The records: Every record behind this note, with a SHA-256 hash per file. Each file the manifest lists is served at its own path beside it.
What this does not show.
- A single product, a single hypothetical feature and 6 briefs. Customer outcomes and later consequences are not yet observed.
- The ceilings use CLI list price estimates. No cash moved on the subscription plan. A session stops after the call that crosses its ceiling, so a crossing call can overshoot.
- The founder did not perform the substantive reads. The reviewer was an agent of a different model family from the reports' authors, reading in a copy of the repository with the letter map and the named run directories removed, retrieving each cited source over the web. No human minutes were recorded, so the human time clause in the pre registered rule was not measured and reads as not met.
Revisions.
This URL is permanent. This table records version numbers and changes.
| Version | Date | Change |
|---|---|---|
| 1.0.0 | 09-17-2026 | First publication. |
More from Research.
The other measurements from the same repository, and the papers they sit beside.
Cite this.
Bartak, J. (2026). A Single Agent and a Research Team on Equal Ceilings. Orbyt Labs Research, version 1.0.0. https://www.orbytlabs.ai/research/single-agent-vs-team
CC BY 4.0. Reuse it with attribution.