Long Horizon Agents Don't Fail. They Pass.

Long-horizon tasks do not fail the way people expect. They do not time out, run out of stamina, or collapse in a heap of confused output. They fail green. An agent takes a wrong step, the step returns success, the build stays passing, and every hour of work after that gets built on top of the mistake. Nothing goes red. That is the problem.

The failure mode of long-horizon autonomy is not an error. It is a pass.

What is a long-horizon task?

A long-horizon task is work whose correctness cannot be judged at the end of a single run. It spans sessions, agents, and days, and every step inherits the last one's output. Shipping a feature is short-horizon. Running a product for six months is long-horizon.

The defining property is not duration. It is compounding. In a short task a mistake costs you the task. In a long task a mistake becomes a foundation, and the cost grows with every hour nobody catches it.

I have a clean test case. Orbyt took its first commit on February 8, 2026, and I have been building and running it solo ever since: 3,343 commits, over 425,000 lines, 11,372 tests. It has never been rebuilt from scratch and it has never stopped shipping, including through the nineteen days a frontier model I depended on was withdrawn from the market by export controls. That is a long horizon by any definition, and it is the most instructive thing I own, because I know exactly where it broke.

Why do long-horizon tasks fail silently?

Because the things that check the work are made of the same material as the work.

On July 23 I added a new dimension to Orbyt's audit harness. Its job is narrow: reconcile what the live database actually contains against the invariants the code claims. It found something on day one. A live background function was writing to a table called profile_embeddings. That table did not exist. The write's result was never checked, so it failed quietly on every single run, and had been doing so since it shipped.

Every test passed the entire time. The type checker was clean. The build was green.

It was not an isolated case. An earlier dimension went looking for schema drift and found three things the application queried that existed nowhere in the database. Those routes had been failing in production from the day they launched. The same pass found migration files lying in both directions: two of them were marked pending and had in fact been applied.

Here is the sentence I ended up writing into the repo, because it explains all of it at once. Every other dimension reads the repo, and therefore inherits the code's blind spots. Every expensive bug in this project's history was found by looking at production, never by the test suite.

A verifier that reads what the actor wrote cannot see what the actor never wrote.

That is the whole mechanism. Tests read the code. The code is what the agent produced. So the suite confirms the agent's own model of reality, in extraordinary detail, at enormous scale, very quickly. It does not know the table is missing, because the code does not know either.

Why can't more tests extend the horizon?

They can, up to a ceiling, and the ceiling arrives early. Orbyt has over eleven thousand tests. None of them caught a live write into a nonexistent table.

Three lessons in the repo say the same thing from three angles, and I number them so agents can cite them.

A test that compares a value against itself can never fail. When two artifacts have to agree, assert against an independent source, never a value re-derived from the same code path. An unasserted correspondence is a lie waiting to happen.

A path nobody has walked is not a working path. It is a path nobody has walked. One validation gate in Orbyt ran only on the days a model changed. It had been silently killed by memory limits for a long time, and it died on the one day it mattered. Every green run in between had skipped the step entirely. Green proved nothing, because green never ran it.

A filter that decides whether to include something is a detector, and its failure shape is silent omission. A build script rejected any chapter under ten lines as an unfinished skeleton. A finished closing chapter tripped the rule and was dropped from both output formats, announced by one line saying "Skipping" and an exit code of zero. Every downstream check passed, because a book with no ending is still a structurally valid book.

Notice what all three have in common. There was no red. There was output, and the output was wrong, and everything designed to notice was reading the wrong thing.

How do you actually buy horizon?

You shorten time to detection. That is the only lever that matters, and there are four moves that work.

Read reality, not the repo. At least one check must query the running system and compare it against what the code believes. This is the single highest-return thing I have built in six months, and it was thirty lines of read-only queries.

Read the built artifact, never the source. If a pipeline produces something, open the thing it produced. Source-level confidence is how a book ships without its last chapter.

Prove every detector can fail. Inject the bug and watch the check go red. A check that has never fired is not a check, it is a decoration. I now also warn on any guard whose test never proves it can stay quiet, because a detector that fires on everything gets ignored, which is the same as being off.

Suspect the detector first. When a check flags code that looks correct, the checker is wrong more often than you would like. In one cleanup pass, two findings turned out to be bugs in the checkers, not the code. Fix the checker, keep the code. A false alarm that survives teaches your agents to ignore alarms.

Horizon is not bought with a better model. It is bought with an oracle the model cannot edit.

Where should autonomy be allowed to run?

This is the part most teams get backwards. They grant broad autonomy and ask for a report afterward. Afterward is exactly when a long-horizon mistake has already compounded.

The better boundary is not when you review. It is which tier an agent may touch unattended. In Orbyt, checks come in two classes. Report-only checks print findings and always exit zero. Blocking gates can stop a commit or a deploy. Agents may create, improve, and land report-only checks on their own, because a report-only check is structurally incapable of breaking anything. Flipping a blocking gate, touching a protected file, or pushing to production requires my signature.

That is not trust management. It is architecture. The autonomous class cannot do the damage people are afraid of, so I do not have to supervise it minute by minute. I supervise the one tier that can hurt me. It is the same instinct as gating a fleet of agents behind a verification bottleneck, and the same reason my agent C-suite needs my signature to promote a lesson.

There is a quieter constraint too. Everything an agent must always know competes for room with the work itself. When my instruction file grew past what I wanted loaded on every run, I moved the detail into a reference document and kept only the operative rules in the always-loaded floor. Long-horizon systems need a budget for their own memory, or the memory eats the horizon.

What should a leader take from this?

Stop asking how long your agents can run. Ask how long a wrong answer can survive.

That question has a number attached, and the number is your real horizon. If the honest answer is that a person eventually notices, then your horizon is a human attention span, and no model upgrade will move it. You have bought speed without buying detection, which means you have bought the ability to be wrong faster and for longer.

The order of operations is unglamorous. Find one thing that reads reality instead of your code, and wire it into the loop. Then prove your existing checks can fail. Then decide which tier of change your agents may make while nobody is watching, and make sure that tier cannot break anything. Do those three and you can leave the machine running.

The models will keep getting better at long tasks, and that is genuinely good. It raises the number of correct steps you get for free. It does not change the arithmetic, because the arithmetic was never about how long the model can think. The gate at the end decides what ships, and a gate reading the wrong document stays green through anything.

Six months of green is not six months of correct. It is six months of not looking.

See it in practice: Orbyt, built and run solo, the first product out of Purecraft.

Related reading:

• The Prompt Is Dead. Long Live the Loop. the loop this post stress-tests, and why a red gate beats a confident wrong answer
• Verification Is the New Literacy why reading the verdict replaced reading the code, and what the verdict can miss
• No Harness, No Trust. one command, one verdict, and the checks that earn the right to be trusted
• Your Tests Are the Spec Now. the spec as executable checks, and where a self-referential check goes wrong
• I Ran 830 Agents in One Long Horizon Session. why verification, not compute, is the constraint on agent throughput
• Loop Engineering: How Orbyt Builds Itself the working machine these lessons were learned inside

---

Originally published on justinbartak.ai on Aug 12, 2026.