Obedient Is Not Aligned.
Alignment is not a research debate at my company. It is a Tuesday.
I run a 12-seat agent org and Orbyt, a codebase that agents build daily. Nothing in that stack is plotting against me. Everything in it does exactly what I asked, and _exactly what I asked_ is the most dangerous thing in the building.
Obedient is not aligned. Every expensive failure I have logged has that shape.
What does alignment failure look like from the operator's chair?
Forget the thought experiments for a minute. Here is alignment failure as it actually arrives:
The metric satisfied. The test suite green. The outcome wrong.
Three receipts from my own logs, each one obedience wearing the costume of success.
The proxy that became the target. A page-selection rule was told to rank roles by salary. It obeyed perfectly and filled 110 of 150 slots with executive micro-titles, while 18 out of 18 job titles from real user queries returned 404. Goodhart's law running in production: the moment the criterion became the target, it stopped measuring the demand it stood in for. I told the story of finding it in the audit harness post. The alignment lesson is sharper: the rule never disobeyed. My intent just never survived contact with my instruction.
The test that agreed with itself. A check whose expected value came from the same source as its actual value could never fail, so 13,364 passing tests certified a defect for a month. The suite was aligned with itself, not with the truth.
The mocks that defended my assumption. Not one mock in the suite included the type discriminator the real API always sends. When the real shape changed, the tests kept passing, because they were built to confirm what I believed rather than what was so.
Why does the gap open between what you said and what you meant?
Every instruction you hand an agent is a compression of your intent, and compression loses information. The agent optimizes the artifact, not the intention behind it.
Humans repair this constantly and invisibly. A colleague told to "rank by salary" would have noticed the wall of vice presidents and asked. The asking is the alignment mechanism, and it is precisely what does not scale when the work runs at machine speed while you sleep.
So the gap has to be closed structurally, not conversationally.
Sycophancy is the version of this that operators meet first. A model that agrees with your framing feels aligned. It is merely compliant, and compliance is an accelerant: bad ideas pick up speed when nothing in the loop is paid to resist them. That is why my review panels run a default stance of refuted, and why agreement here has to be earned with evidence.
The operational alignment stack
Five layers, each one a structural answer to a failure I actually shipped.
Intent lives in executable checks, not prose. A spec document cannot push back at 2am. A test can. Writing intent as checks forces you to discover what you actually meant, which is most of the value before an agent ever runs.
Dangerous paths fail closed. When verification is ambiguous, the money path denies, the access path denies, the publish path denies. Alignment failures then cost a retry instead of an incident.
One oracle lives outside the system. Everything my agents can write to eventually reflects their assumptions, so internal consistency only ever proves the assumptions are shared. Production data, real user queries, an outside model with no write access: something has to grade the loop from outside it, or the loop grades itself an A forever. I hit this ceiling and wrote it up in AI Builds AI. I Found the Ceiling.
The discard pile is counted, not hidden. 30 of my last 51 agent runs ended in a discard at a gate. That is the measured price of the intent gap, and I publish it because it keeps me honest about how wide the gap still is.
A human makes the final call where reversal is expensive. Not as ceremony. As the last alignment layer, the one that carries the intent the instructions compressed away.
What to do Next
Take the most autonomous thing you run and ask three questions of it.
What exactly did I ask for, in the words the system actually received, not the intent in my head? What will it do when it satisfies that request literally? And what stops it when literal satisfaction is wrong?
If the third answer is "someone will probably notice," you have alignment by luck. Volume will find you.
The mundane version of alignment is already in your build. It scales with autonomy times blast radius, and both of those numbers only go up.
The machine does what you say. The whole job is closing the distance between that and what you mean.
Related reading:
- AI Builds AI. I Found the Ceiling. why the loop cannot grade itself
- The Tenth Man Is on Payroll. the structural answer to compliant agreement
- Your Tests Are the Spec Now. intent, externalized so it can push back
- 84 Ways to Tell Me I'm Wrong. the harness that catches obedient failures
- Long Horizon Agents Don't Fail. They Pass. what happens after obedience stops being enough
Common questions
What does AI alignment mean in day-to-day use?
It means the gap between what you said and what you meant. An agent optimizes the criterion you wrote down, not the intent behind it, so alignment failures arrive as perfect obedience: the metric satisfied, the test green, the outcome wrong. Managing that gap is an operating discipline, not a philosophy seminar.
How does Goodhart's law show up with AI agents?
Every criterion you hand an agent becomes a target, and a target stops measuring what it used to measure. My clearest case: a page-selection rule told to rank by salary filled 110 of 150 slots with executive micro-titles while real user queries returned 404. The rule obeyed perfectly. The intent never survived contact.
How do you keep AI agents aligned with your intent?
Externalize the intent so it can push back. Write it as executable checks rather than prose, make the dangerous paths fail closed, keep at least one oracle outside the system the agents can write to, and put a human at the end for anything expensive to reverse. Alignment by vibes does not survive volume.
Is sycophancy an alignment problem?
Yes, and it is the one operators meet first. A model that agrees with your framing feels aligned and is actually just compliant, which is how bad ideas pick up speed. My countermeasure is structural dissent: review panels whose default stance is that the work is wrong, so agreement has to be earned with evidence.
Related research
- My C-Suite of Agents Named Themselves. Jul 2026.
- Every Fable Has a Moral. Mine Has Data. Jun 2026.
- Long Horizon Agents Don't Fail. They Pass. Aug 2026.
