Self-Healing Is a Euphemism.

Self-healing sounds like the system got smart. It did not. In my stack it is roughly 20 boring, bounded repairs, each one a pre-decided fix for a failure that already happened, each one logging the fact that it ran.

That last clause is the whole post. A repair nobody counts is not resilience. It is a defect with a subscription.

Healing that does not report is just a failure you stopped seeing.

What does self-healing actually look like?

Nothing like the phrase suggests. Here is the real inventory from Orbyt, in the plainest terms I can write them.

An auto-heal pass runs on app load, throttled to once every 12 hours. It repairs orphaned references and bidirectional links that fell out of sync.

An offline write queue catches writes that failed while the connection was gone and flushes them on reconnect. It dedupes by table and column, caps at 5 retries and 200 entries, and routes anything exhausted to a failed-writes store instead of dropping it.

Row-level security errors get exponential backoff at 2, 4, and 8 seconds, then stop and log that the write will sync on the next load.

A reconciliation pass on reconnect, throttled to 60 seconds, compares per-column timestamps and pushes only the columns that are locally newer. Not the row. The columns.

On the data side, ingestion quarantines anomalous rows out of the live table rather than rejecting them, flags disagreement when any source deviates more than 25% from consensus, and quarantines any tuple with fewer than 5 samples so a thin slice never reaches a customer.

None of that is intelligence. All of it is a decision somebody made once, in advance, about what good looks like.

What separates healing from improving?

Healing restores the state the spec already describes. Improving changes the spec.

That line is the entire safety boundary. Restoring a known-good state can run unattended because the target is known, the action is bounded, and the blast radius was measured before it ever shipped. Changing what the product is has no known target, and confidence is not evidence.

This is why I do not let a system that can repair itself also decide what it should be. The loop cannot grade itself, and a system that can rewrite its own definition of correct will eventually find the cheapest definition.

A machine may restore the standard. It may not move it.

What makes a repair safe to run while you sleep?

Six properties. All six, not five.

Observable. Every run emits a structured line, including the no-ops. A repair that only logs when it acts hides its own frequency, which is the number that matters most.

Idempotent. Running twice equals running once. Crons double-fire under retries, and idempotency is what keeps a retry from becoming a second incident.

Bounded. Capped retries, capped batch size, capped duration. My write queue stops at 5 retries and 200 entries precisely because an unbounded repair loop is a denial of service you wrote yourself.

Fail open. A broken repair must never break the customer path. My request-log writer fails open by design: if the log write dies, the customer still gets their response, and the failure lands in stderr where it belongs.

That one deserves care, because everything dangerous in my stack fails closed. The rule is not universal, it is directional. Money, access, and publishing fail closed. Bookkeeping fails open. Getting those backwards is how a logging outage becomes an outage.

Overridable. The operator can switch a repair off without a deploy.

Tested. Every repair carries example tests and a property test for its invariant, inside the 11,372 that run on every change.

Where does self-healing go wrong?

It goes quiet.

A repair that always succeeds and never reports is indistinguishable from a system that never breaks. The root cause keeps its funding, the bandage keeps its schedule, and the org congratulates itself on reliability it did not build.

So the metric is not whether repairs succeed. It is how often they fire. A heal that runs 4 times a week this quarter and 40 times a week next quarter is telling you something specific, and it is not that the system is getting better at healing.

I treat rising heal counts the way I treat rising discard rates: as an argument against me. 35 of my 58 recorded failures are now permanent mechanized guards precisely because a repair is a stopgap and a guard is a fix. The gap between those numbers is the honest part.

What to do Next

List every automatic retry, fallback, and cleanup job in your system. Most teams find between 10 and 30 and are surprised by half of them.

For each one, answer four questions. Does it log every run, including the times it did nothing? Is it bounded? Does it fail in the direction that matches its blast radius? And when did somebody last look at how often it fires?

Then pick the one that fires most and go fix what it is repairing. That repair is not resilience. It is a receipt for work you deferred.

Self-healing is only a feature when you can count it. Otherwise it is a place your bugs go to live quietly.

Related reading:

• 84 Ways to Tell Me I'm Wrong. the guards that turn a repair into a permanent fix
• AI Builds AI. I Found the Ceiling. why a system cannot be trusted to redefine its own standard
• Safety Is a Default, Not a Debate. which way each path should fail
• Verification Is the New Literacy reading the verdict instead of the diff
• Your Linter Was the Prototype. the build-time half: code that repairs code, and what it is never allowed to touch
• Heal What You Can Prove. the artifact half: systems that rewrite their own code and prose, and the never-heal list