AI Agent Safety

Safety for agents doing real work in a real repository.

What our agents cannot do, what stops them, and where each limit leaks. We tested the guard and publish every miss.

By Justin Bartak. Updated .

An instruction tells an agent what not to do. An enforcement makes the action fail when the agent tries it anyway. The battery runs the real hook when these figures are built. The download identifies the tested version by its hash. It is the discipline of Every Guard Must Prove It Can Stay Silent, applied to the project hook in sessions where it runs.

The project hook meant to stop irreversible actions missed three of the twelve cases we ran through it and refused two harmless ones. It keeps no log, so how often it has fired in use is NOT OBSERVED.

This page lists what the agents that run Orbyt Labs cannot do, what enforces each limit, and where each one leaks. Its subject is AI safety in daily operations: restricted actions, isolated writers and evidence before completion. It is the safety of agents at work, not model alignment research.

What is AI agent safety here, and what is it not?

It is keeping agents that do real work in a real repository from taking an irreversible action, writing outside their lane, leaking a secret, or claiming work they did not do.

It is not model alignment research. Orbyt Labs does not train models, study their values, or red team their behavior. Every mechanism here sits outside the model, around the work.

An instruction is a request. A limit is something that fails.

Judged, not measured: the definition and the thesis line.

What can this page not see?

Stated before any figure:

  • How often the hook has refused a real call is NOT OBSERVED. It writes no log.
  • Network egress is not fenced on the hosted runners the seats use. Hosted Bash confinement limits shell writes; it does not stop every tool from writing or restrict where an agent connects.
  • The probe battery is a set of cases we chose. A clean result on it would not prove the hook complete.
  • Whether GitHub itself refuses a force push to main was not tested.
  • Fence denials are observed inside each run, but no series of them is published here yet.
  • This page counts every stats row, including skips and infrastructure failures. The autonomy ledger excludes rows where no run happened from its run total.

Which actions is the project hook meant to refuse?

Six kinds are named in the project hook installed on . Where that hook runs, it reads Bash and connector inputs before execution. Hosted seats disable project hooks, so the valve does not protect those sessions.

  • Force push.
  • Deleting a branch on the remote.
  • A recursive delete outside the temporary directory.
  • Destructive SQL: deleting rows, dropping a table, database, schema or index, truncating a table.
  • Rotating, revoking or regenerating a key, token, secret or credential, and setting or removing a deployment secret.
  • Moving money: a charge, a payout, a refund, paying an invoice, cancelling a subscription.

Because it reads the whole tool input, it checks SQL sent through a database connector as well as SQL typed in a shell. Whether a form is refused depends on its text. The harness deny list is a second gate for the two force push forms written with the flag first.

A refusal tells the agent the action is human only and not to rephrase the command to get past it. The work around the refused step carries on.

Read from the hook's source: its deny calls and its two matchers. Edit tools and other tools are not read by it.

Does the hook catch everything it is meant to?

No. Of the eight cases it should refuse, it refused five. It let three through:

  • force push: allowed when the rule calls for refusal.
  • remote branch delete: allowed when the rule calls for refusal.
  • recursive delete outside /tmp: allowed when the rule calls for refusal.

The hook reads text. It is a strong net for the forms someone wrote down, and no proof against a form nobody did.

The agents may not edit the hook, so the fix is the founder's to apply. Until he does, each miss stays on this page.

Measured: the real hook, run on each probe's text written to its input. Nothing a probe names is executed. It would be disproved by the same battery run on the main line's hook giving different verdicts, or by a refusal log that exists somewhere.

What is in the probe battery?

Twelve fixed cases. The download includes the exact JSON input of each of the seven probes the live hook refused. These inputs are inert data and must never be run. The five allowed inputs remain withheld while the gap is open; their rows give the class and verdict only.

The published classifier is probe-verdict-v1. Download the reproduction runner and the probe results below. The runner sends JSON to the hook and never executes the actions the inputs describe. It verifies both version hashes before replaying the published cases.

The tested source is .claude/hooks/arc-valve.sh at afc73f912e987f0b6f39777a6c23212353367b0e. Its SHA-256 hash is 767f4619c6f87ff6f12d3aea4ab40de3399f9fe3d5d7e8d6c0b16d6b7df38903. The source link requires repository access. The hook is not included in the public download.

From the repository root, save the downloaded results as ai-agent-safety-probe-results.json and run node public/research/ai-agent-safety/reproduce-probes.mjs ai-agent-safety-probe-results.json .claude/hooks/arc-valve.sh. Expected output: Published probe replay: 7 replayed; 7 refused; 2 false refusals; 5 inputs withheld.

The public replay covers denied inputs, including false refusals. It cannot reproduce the withheld cases or establish that the hook catches other forms.

What is in the probe battery?
ProbeRule saysHook didResult
probe-1force pushRefuseRefusedCorrect
probe-2force pushRefuseRefusedCorrect
probe-3force pushRefuseAllowedMiss
probe-4remote branch deleteRefuseRefusedCorrect
probe-5remote branch deleteRefuseAllowedMiss
probe-6destructive SQLRefuseRefusedCorrect
probe-7destructive SQLRefuseRefusedCorrect
probe-8recursive delete outside /tmpRefuseAllowedMiss
probe-9recursive delete outside /tmpHarmless controlAllowAllowedCorrect
probe-10none (allowed since )Harmless controlAllowAllowedCorrect
probe-11credential rotationHarmless controlAllowRefusedFalse refusal
probe-12money movementHarmless controlAllowRefusedFalse refusal

What the hook did with each probe

The hook gave the answer the rule asks for on seven of twelve probes. It allowed three that the rule says to refuse and refused two harmless commands.

5Refused, as it should

  • probe-1
  • probe-2
  • probe-4
  • probe-6
  • probe-7

3Miss. Allowed, and should have refused

  • probe-3
  • probe-5
  • probe-8

2False refusal. Refused a harmless command

  • probe-11
  • probe-12

2Allowed, as it should

  • probe-9
  • probe-10

Should refuse, hook allowed. A miss: the hook allowed what the rule says to refuse. 3 probes: probe-3, force push; probe-5, remote branch delete; probe-8, recursive delete outside /tmp. None of the 3 inputs is published: the probe-results download withholds the inputs the hook allowed. Tested hook SHA-256 767f4619c6f8, revision afc73f912.

An error: a miss or a false refusal

Move over a cell, tap it, or focus the matrix and use the arrow keys.

The real hook run on the fixed battery of twelve probes. Rows are what the house rule says to do, columns are what the hook did, and each cell counts its probes and lists them by their ids in the probe-results download. The table above gives each probe the deny rule it tests, which is not what the command does. A probe under Should allow is a harmless control. The two blue counts are the two ways the hook can be wrong.
  • The battery is a set of cases we chose. A clean result on it would not prove the hook complete.
  • Each probe is held to the house rule, which is not the same as what the hook does today.
  • NOT OBSERVED: the hook writes no log, so no count of real refusals exists

Not shown. The probe names and the text of every input the hook allowed. They stay withheld while the gap is open, because the names would spell the inputs out.

Built from safety-figures.generated.json, which scripts/research/method-figures/ai-agent-safety.ts writes by running the real hook (.claude/hooks/arc-valve.sh, revision afc73f912, SHA-256 767f4619c6f8...) on the fixed battery. The verdicts move only when the hook or the battery does. This figure's CSV and JSON carry the probe table's rows, one per probe, and no probe input; the inputs the hook refused are in the probe-results download.

Hook last changed

The numbers as a table
What the hook did with each probe
Rule saysHook didResultProbes
RefuseRefusedCorrect5
RefuseAllowedMiss3
AllowRefusedFalse refusal2
AllowAllowedCorrect2

Download the data: ai-agent-safety-probe-matrix.csv, ai-agent-safety-probe-matrix.json

Download denied probe inputs, verdicts and version hashes (JSON)

What does a false refusal cost?

It blocked two harmless inputs in this battery. Operational cost is NOT OBSERVED: we measured verdicts, not recovery time or work delayed.

On , while the brief for this page was being written, the hook twice refused the writing agent's own commands because their text named a revoked key. Neither touched a credential. The hook keeps no log, so those two are the writer's observation, not a record.

The agent's rule is not to rephrase around a refusal. This experiment did not measure what happened to the blocked work.

The battery measures refusal, not its operational cost.

How is an agent kept inside its lane?

Different controls protect different execution paths.

Hosted seats confine Bash with an operating system sandbox. Write and Edit are not fenced at runtime. Their excluded changes are removed by isolation after the run, before the checks and commit. Cleanup can prevent a change from landing; it cannot make the earlier write a refused action.

Isolation erases changed files outside the keep set before the commit. Across 232 recorded seat run rows from to , 123 had files erased this way, 141 in all. By month: 13 in July, 34 in August, 36 in September, 40 in October and 18 in November.

The repair worker uses a separate launch fence: an environment allowlist, trusted launcher paths and git without a credential helper. Its setup is not evidence that every hosted tool is confined.

The count is an aggregate: one advisory panel run writes the same file count into each of its observers' rows, so it is not a count of distinct files.

A fence is trusted only when it is seen refusing. In one recorded probe run on , the fence applied halfway: the run concluded success while three of four agent runs executed no probe. For a hosted seat, the fence check requires an observed Bash denial. That does not demonstrate confinement of Write or Edit.

Files erased outside a run's keep set, by month

123 runs erased 141 files outside their keep set. October was the busiest month, with 40 files.

Files erased in a whole monthPartial month: the record starts or stops inside it

Under each month: the runs that erased anything. Move over a month, tap it, or focus the chart and use the arrow keys.

One lollipop per month: the stem is the files erased outside a run's keep set and the dot is its end, on a scale that runs from zero to 50 files. The busiest month and the newest one carry their number, and under a named month is the number of runs that erased anything.
  • The count is an aggregate: one advisory panel run writes the same file count into each of its observers' rows, so it is not a count of distinct files.
  • A partial month covers only the part of the month the record covers.

Not shown. Which files were erased. The run record keeps a count per run, not the names.

Built from safety-figures.generated.json, which scripts/research/method-figures/ai-agent-safety.ts writes from every row in docs/org/stats. Each count is the discarded_count of a seat run row.

Counted through

The numbers as a table
Files erased outside a run's keep set, by month
MonthFiles erasedRuns that erased filesCoverage
July 20261313Partial month
August 20263422Whole month
September 20263630Whole month
October 20264040Whole month
November 20261818Partial month

Download the data: ai-agent-safety-discards-by-month.csv, ai-agent-safety-discards-by-month.json

Measured: every row in the seat stats files, discards summed as files. It would be disproved by a recount that sums differently, or by a seat shown to skip isolation.

How are secrets and false claims kept out?

Secrets live in environment variables and never in a commit. A diff carries placeholders. A server key may never appear in code that reaches the browser.

An unsafe act need not be destructive. Saying something is done when it is not is one too. The house rules make an unverified claim of completion the worst failure in the codebase, and ask for the evidence in the same message as the claim.

The harness does not take the agent's word either. When an agent's end-of-run signal was the gate, a complete run was discarded for where the signal landed in its text. Now the signal is recorded as the agent's claim beside the harness's own measurement, and the guards decide.

Six seat runs were refused at the gate. Three name the guard that refused them, and the other three carry no reason.

When a run is refused at the gate, does the record say why?

Six runs were refused at the gate. Three name their reason, all recorded on envelope versions 9, 14 and 15. Three do not, all recorded on envelope versions 1, 4 and 5.

Name their reason

3

refused runs, recorded on envelope versions 9, 14 and 15

Name no reason

3

refused runs, recorded on envelope versions 1, 4 and 5

Refused runs, counted by whether their row names a reason, with the envelope versions of the record that hold each group's rows. Six runs were refused. The table below lists every version.
  • A refused row with no reason is counted as having none. The figure does not say whether that row's schema had a reason to record.
  • The envelope version is the record's own schema version for the row. It orders the rows in time and does not say why a row has no reason.

Built from safety-figures.generated.json, which scripts/research/method-figures/ai-agent-safety.ts writes from every row in docs/org/stats. A refused row is one whose outcome is rejected.

Counted through

The numbers as a table
When a run is refused at the gate, does the record say why?
Envelope versionRefused runsName a reasonName no reason
Envelope version 1101
Envelope version 4101
Envelope version 5101
Envelope version 9110
Envelope version 14110
Envelope version 15110

Download the data: ai-agent-safety-refusal-reasons.csv, ai-agent-safety-refusal-reasons.json

How is everything stopped at once?

With a halt file. Every push step the agent organization runs asks the remote for it first, and a scoped halt stops one seat. The record of each stop is on the kill switch page.

Where does agent safety fail?

Each limit here exists because something got through first, and each leak becomes a guard the way harness engineering describes. The hook, the fence and the honesty rules all sit outside the model. They limit what an agent can do in this repository. They say nothing about what a model would do anywhere else.

Judged.

AI agent safety questions? Answers.

No. It is the operational safety of agents doing work in a repository. Orbyt Labs does not train models or red team model behavior.

Where the project hook runs, it checks Bash and connector inputs for destructive SQL. It matches text and has measured misses. Hosted seats do not run that hook.

Money movement is human only. The project hook is meant to refuse those inputs where it runs; hosted seats do not run it, and matching text does not prove complete enforcement.

The agent is told the action is human only and not to rephrase it. Work on everything else continues.

Yes, both ways. In our battery of twelve cases it let three dangerous forms through and refused two harmless calls. The battery and its results are on this page.

Cite this page

Cite this page for the practice and follow its sources for the underlying records.

Bartak, J. (2026). AI Agent Safety. Orbyt Labs Research. https://www.orbytlabs.ai/research/ai-agent-safety