The Test Suite Became Orbyt's Only Living Specification.

The thesis.

When an AI agent can ship changes faster than a person can update a document, the document stops being trustworthy. The article's claim is that executable tests, not prose specs, become the artifact a codebase actually obeys, because only an executable artifact can verify itself on every commit. A spec written Monday can describe a system that no longer exists by Thursday, and nothing forces it to catch up.

What was found.

The author reports that Orbyt, built solo in 32 days, is now over 425,000 lines governed by 11,372 tests, with no prose document describing what the system is supposed to do. He states there is no separate specification he maintains as source of truth. The tests sit inside what he calls a 35-dimension audit harness, run through a single command that checks build, types, lint, fifteen locales, the iOS app, the Safari extension, a security battery, and accessibility. He states that production readiness is decided by whether that harness passes, not by whether a model reports a task as done.

He also reports switching frontier models mid-session twice in June 2026, once by choice and once because of export controls, and states both swaps were uneventful because the test suite held the contract rather than the model.

The evidence, as stated.

The evidence offered is a first-person account of one project. The author asserts, without citing external data, that a prose spec is a description of intent while a test is a description of behavior that fails when the system stops matching it. He asserts that high line-coverage numbers on the wrong paths are theater, and that the real measure is whether every path an organization cannot afford to break has a check that fails closed. No coverage percentage, defect count, or incident count is given for Orbyt itself. The claim that both June 2026 model swaps were "uneventful" is asserted without a supporting metric, such as tests run, tests broken, or time to resolution.

How this was done.

The article describes a practice, not an experiment. The stated method: encode every non-negotiable behavior as a test before letting an agent work in that area; require the agent to write a test alongside every feature and reject changes that ship behavior without a corresponding check; wrap all checks in a single-command harness so verification is one action rather than a checklist; and make critical paths fail closed, so an unverified state denies access rather than granting it. On Orbyt specifically, the author states the agent writes the test alongside the feature, and that he treats the test, not the diff, as the artifact he trusts. He gives one concrete example of the fail-closed principle: subscription gating on Orbyt denies access when a check breaks rather than granting it.

What it implies.

If the account is accurate, it implies that a test suite functioning as the specification survives changes to the underlying model, because the contract lives in the checks rather than in any one model's behavior. It also implies that measuring AI-assisted development by lines of code or coverage percentage is measuring the wrong thing, and that a more relevant unit is the count of checks that pass without a human in the loop, particularly on paths where an error would cost a customer, a fine, or trust. The article's recommendation for other organizations is to identify the paths where being wrong is expensive, encode each as a fail-closed test, and treat a passing harness, not a model's self-report, as the definition of done.

What this does not establish.

This is a single practitioner's account of a single project he built and controls, not an independently verified case study, an audit, or a comparison against teams that kept prose specs. No before-and-after data is given: there is no baseline showing how many defects a prose-spec approach would have missed on the same codebase, no incident log, and no measurement of how much time writing 11,372 tests took relative to writing and maintaining a spec document. The claim that both June 2026 model swaps were uneventful is not quantified, so the reader cannot assess how close either swap came to a real failure. The 35 dimensions of the audit harness are named but not fully enumerated or independently described, so their actual rigor cannot be assessed from the article alone. The author states plainly that the harness covers 35 dimensions and nothing outside them, and that the uncovered gaps are where the next incident would originate, which is itself an acknowledged limitation of the method rather than a resolved one. Finally, the argument that tests are more durable than prose specs is presented as a general claim about AI-native development, but the only supporting evidence is one founder's report about one product he built alone.