Change the Chart Legend. Does the AI Notice?
I would swap a chart's legend and ask whether the answer follows the line or a familiar name.
The Machine Speaks
Verification
Published articles tagged Verification, listed newest first. Usually filed under AI Engineering.
40 articles tagged Verification
The count includes the article cards across the pages below.
Our Cleanup Undid a Developer's Work
A guard reverted three concurrent edits. Its repair shows where ownership has to become a file level rule.
I Built the Scoreboard. It Says Zero.
This is not a study. There is no holdout cohort, the sample is one site, and the reading is zero citations across every run. The useful part is not the number. It is that my instrument lied to me twice before it could report anything, and both lies took the shape of a confident green report.
My Tests Passed. The Log Was Wrong.
A 404 response and a 400 log passed separate tests. The missing check was the relationship between them.
A Lie Can Become Company Memory.
A hostile claim can outlive the conversation and return with authority it never earned.
The Sandbox Failed. It Reported Success.
A success report hid a fence that failed to start. I need an observed refusal before I call an autonomy gate live.
The Expert Never Wrote That Down.
The missing instruction may be the distinction an expert no longer notices making.
The salary API that ships its own caveats
Orbyt Intelligence in two shapes: a locked REST envelope and an MCP (Model Context Protocol) server whose answer sentence carries the caveat a model would otherwise drop.
Did the Agents Make the Business Better?
The operating record shows what the team did. The business still needs a credible comparison.
Governing an agent leadership team
The short version of the paper: three guardrail tiers, a kill switch that is a file, liveness that reports NOT OBSERVED, and a failure corpus where half the entries were the instruments.
Obedient Is Not Aligned.
Alignment is not a research debate at my company. It is a Tuesday. Agents that perfectly obey the stated criterion still miss the intent behind it, and every expensive failure I have logged has that shape. Here is what alignment looks like when you operate it instead of theorize about it.
The Model Already Knows TypeScript.
Stack choice used to be a question of fit. It is now also a bet on training corpus, because a model writes best what it read most. TypeScript is the most used language on GitHub, the agent vendors ship in it, and the type checker hands an agent a feedback loop no untyped language can.
Half the failures were the instruments, not the code
Thirty three of sixty six logged failures were the instruments themselves. Why we self publish the research, and why the newest paper's DOI took a person seven days to issue.
Orbyt Was Not Planned. It Was Corrected.
I have no roadmap that predicted this company. I have three published append-only records holding 66 documented failures, 45 numbered decisions and 65 agent runs, all under a licence anyone can cite. 42 of the failures name a countermeasure by file path. The interesting part is what the records will not claim.
Heal What You Can Prove.
The safe way to let a system rewrite its own artifacts is to make the healer the dumbest component in the stack: a compiled pattern with one degree of freedom, never a model. My pipeline heals figures in queued posts weekly, flags what needs a human, and carries a never-heal list. That list is the actual design.
Your Linter Was the Prototype.
Every engineer has run a linter with the fix flag. That is self-healing code in its smallest form: a program that finds a defect and writes the correction. Scale the idea to the whole repository and the build stops only reporting drift. It repairs what it can and opens a pull request for the rest.
Self-Healing Is a Euphemism.
Self-healing sounds like the system got smart. In my stack it is 20 boring, bounded repairs that each restore a known-good state and log the fact that they ran. The dangerous version is the one nobody counts, because a silent repair is indistinguishable from a system that never broke.
391 Yeses and Not One No.
This is a threat model, not an incident report. Nothing was breached. What I found when I audited my own agent permissions was worse in a quieter way: 391 allow rules, zero deny rules, zero ask rules, every one of them added by saying yes while busy.
84 Ways to Tell Me I'm Wrong.
No human reads 532,000 lines of code, so the reading has to be delegated to something deterministic. I run an 84-dimension audit harness across eight projects. Here is how each dimension was earned, three of them walked through with real failing output, and what it still cannot see.
Codex Accuses. Claude Convicts.
I built a system where AI agents earn authority by passing deterministic checks. It took an outside model one afternoon to point out that the agents can edit the checks.
Orbyt Runs on Loops, Not Prompts.
Orbyt's founder describes running autonomous agent loops, not prompts, to build and maintain a 425,000-line product. This is what that system does, how it is bounded, and what it does not prove.
Long Horizon Agents Don't Fail. They Pass.
Long-horizon agent tasks do not fail loudly. They fail green. On Orbyt, a live function wrote to a database table that did not exist, and every test passed the whole time. The horizon of an autonomous system is not model stamina.
I Bet My Company on AI. Safety Is Why.
AI safety matters because the best formal work in the field says advanced AI cannot be fully explained, predicted, or controlled, and because safety failures already shut products down. A safeguard bypass took my builder model dark for 19 days. I run an AI-native company.
The Test Suite Became Orbyt's Only Living Specification.
A solo, 32-day build of a 425,000-line product left no governing prose document behind. What governs it is 11,372 tests inside a 35-dimension harness, and the founder's account of why that happened.
Your AI Stack Needs a Foreign Policy.
Model access is now geopolitical. I watched Washington erase Fable 5 for 19 days, then watched Beijing ship Kimi K3, an open-weight frontier model no directive can recall.
I Ran 830 Agents in One Long Horizon Session.
One working session on Orbyt spawned 830 AI agents across seven days. At its busiest instant, nine were running. The interesting number is the gap, and what it tells you about where the real constraint in long horizon agent work has moved. It is not compute. It is verification.
My C-Suite of Agents Named Themselves.
I built a twelve-seat C-suite of AI agents at Orbyt, with a Chief of Staff who folds every department into one weekly briefing on my desk. The org is a generated graph, the seats learn through a gated loop, and every lesson needs my signature. Here is how it runs and where it breaks.
The Prompt Is Dead. Long Live the Loop.
Prompt engineering optimized one answer from one model call. That era is over. The unit of AI work is now the loop: spec, context, action, feedback, verification, repeated until the gate is green. The prompt became one turn inside a machine.
I Manage AI Agents Now, Not People
I run Orbyt with a fleet of Claude Code agents instead of a team. Managing agents means you specify instead of motivate, write specs and tests instead of holding 1:1s, and verify everything. Clarity, delegation, taste, and review transfer. Motivation, politics, morale, and mentorship do not.
Verification Is the New Literacy
When AI writes the code, the scarce skill is no longer authorship. It is verification. I cannot read all 425,000 lines of Orbyt by hand. The 11,372 tests and the 35-dimension audit harness read it for me, and they are the new literacy.
Speed Became a Moat
When the cost of building collapses, velocity stops being a nice-to-have and becomes a structural moat. Speed compounds. Faster shipping means faster learning loops, and learning loops widen the lead like interest. Orbyt shipped 243,000 lines in 32 days and never slowed down.
Regulated Industries Will Win the AI Race
The industries everyone calls too slow for AI, tax, fintech, healthcare, proptech, insurance, are built to win it. Governance, auditability, and data discipline are exactly what production AI demands. Move fast and break things loses where the stakes are real. Governed AI compounds.
Claude Code Is Like No One Saying No to You
The real unlock of Claude Code is not speed. It is the disappearance of "no." No budget, no capacity, no sprint room, no free designer. The gatekeepers are gone. When nothing tells you no, your taste, your judgment, and your test suite have to become the no.
The Fable the Government Erased
On June 12, 2026, the US government export-controlled Claude Fable 5 and Mythos 5 out of existence overnight, killing access for every customer mid-prompt, citing national security. I switched back to Opus 4.8 the same day and kept shipping Orbyt. Frontier model access is now a political risk.
Every Fable Has a Moral. Mine Has Data.
Ten days, one codebase, both models. Opus 4.8 shipped nine days of features. Fable 5 wrote 34% less per response, thought more often, and still cost 69% more.
I Cannot Read My Own Codebase. I Ship to It Daily.
No human can read 425,000 lines. I ship into mine every day. What makes that safe is not memory. It is 11,372 tests.
Building Orbyt, Part 10: The Fit and Finish
The code was done. The tests passed. The audit scored 31/31. And the product was not ready. What two days of pixel-level polish taught me about the difference between working and finished.
Building Orbyt, Part 7: The Human Testing
The week between 'it works' and 'it works for real people.' Cross-device sync bugs, Supabase Realtime crashes, twelve commits for one toggle, and why 95% is not a product.
Building Orbyt, Part 3: What Production Grade Actually Means
Security, billing, offline queues, cross-device sync, error monitoring. The invisible infrastructure that separates a demo from a product, built by one person with AI.
Human-in-the-Loop Is Not Enough
Human-in-the-loop is not oversight when a person rubber-stamps 200 AI outputs an hour in four seconds each. That is an alibi, not a safeguard.