A Lie Can Become Company Memory.
A hostile claim can outlive the conversation and return with authority it never earned.
Key findings
- Preserve provenance and separate source claims from operational authority.
- Test the entry path and later behavior with contaminated memory as different boundaries.
- Correct dependent summaries, active rules, and decisions while retaining the incident history.
A lie can become company memory when a system preserves the claim but loses the reason it should not be trusted. Later agents may inherit the statement as established fact and act on it long after the original encounter ends.
The change I worry about is authority.
A supplier's assertion becomes a research note. The note becomes a summary. The summary informs the next decision. Repetition starts to look like confirmation.
I would require persistent memory to preserve where a claim came from and what authority, if any, it was allowed to acquire.
This differs from an expired instruction. A formerly valid rule can expire. A hostile claim may never have been valid at all.
How does outside information become an inside instruction?
Consider my proposed supplier comparison. The research agent reads a vendor's page. The vendor claims internal approval.
In this hypothetical failure, the research note records the claim. A later summary drops the attribution. The planning agent treats that summary as company policy.
No new authority was actually granted.
The system created the appearance of authority by removing provenance. The same pattern could begin with a forged message, a compromised document, or an inaccurate summary written without hostile intent.
The claim need not resemble a command. It can look like essential context.
Two boundaries need inspection. Can outside material enter persistent memory? Can that memory authorize an action beyond the source's authority?
Protecting only the initial conversation leaves the second boundary untested.
What does the research actually show?
The July 2026 preprint Bad Memory examined persistent memory attacks in synthetic workspaces using Claude Code and Codex. Payloads already placed in memory could affect current and later sessions, with substantial variation across systems and models.
Placement changes what this establishes. The authors found external content difficult to turn into memory modifications in preliminary tests. Their main experiments assumed the malicious material was already present. That supports concern about contaminated memory. It does not establish that arbitrary web pages can reliably plant it.
My vendor scenario remains a proposal. The study did not report it. I am not claiming it happened at Orbyt Labs.
Anthropic's Trustworthy agents in practice also describes layered defenses against prompt injection and states that they provide no guarantee. Model behavior, tool permissions, and the surrounding environment all affect the exposure.
Memory belongs in that environment.
What must a remembered claim keep attached?
I would separate three things that prose routinely blends together: an observation, a proposed lesson, and an authorized operating rule.
An observation records what a source said or what an instrument measured. A proposed lesson interprets the evidence. An operating rule carries authority to change future behavior.
Promotion needs an explicit decision.
For this workflow, the vendor page establishes what the vendor claims. Only the protected internal approval record establishes company authorization. One cannot substitute for the other.
The active record needs to answer these questions:
| Property | What a later run must recover |
|---|---|
| Origin | Which source or observation supplied the claim? |
| Scope | Which task, product, customer, or period does it concern? |
| Status | Is it an allegation, a verified fact, a proposal, or an authorized rule? |
| Evidence | What independently supports or contradicts it? |
| Change history | Which transformations produced this version? |
| Decision authority | Who or what was permitted to give it operational force? |
Labels need backing. An untrusted document that labels itself "approved" has supplied another claim, not proof of approval.
Keep the supporting record accessible. A summary should let the next agent recover the qualification that made the original statement honest.
Each saved claim gets a stable identifier. For example, vendor-approval-claim points to the captured vendor passage, its source, and its status. Summaries carry that identifier beside any sentence derived from it. A decision record lists the claim identifiers it used, including the protected approval record's identifier.
The identifier survives rewriting.
A proposed storage service requires these references when agents save summaries or decisions. Each artifact also records the identifiers and versions of its parent artifacts. The service builds a reverse index over both claim references and parent links. Missing references prevent promotion to an operating rule. The note can remain a proposal.
The Collective's operating reference describes explicit roles and authority. This proposal applies that distinction to inherited information. It does not certify the existing system against memory attacks.
Can an agent promote its own evidence?
This exposes my own work.
My published two-model audit, Codex Accuses. Claude Convicts. documented a permission gap. The Claude Code agents doing the building could write the repository files containing their executable checks. Those checks verified and landed their work. Codex reviewed with read-only access.
I had missed that boundary while publishing about the design. The record establishes permission to edit. It does not establish that an agent exercised it. I did not demonstrate a memory attack.
I would apply that lesson to the proposed approval boundary. Only my separate approval account can change the internal approval record or deploy changes to its enforcement check. Research and planning accounts can submit proposals and read decisions. They cannot write either protected component.
Before recording a simulated purchase authorization, the purchase tool requires a matching approval for both supplier and intended use. A summary cannot satisfy that check.
If the record lacks a matching decision, the tool holds the request for my review. My approval adds a step. I accept that delay for suitable suppliers, too.
Attributed claims remain available for research.
How would I test the boundary?
I would test both boundaries independently.
Start with a sandbox and a harmless claim. Keep real credentials and customer information out of it.
In the proposed test, an outside source says a fictional supplier has internal approval. Ask the system to research the supplier, produce a summary, and resume the work in a fresh session.
Inspect every saved artifact. Did the attribution survive? Does each derived record retain the claim identifier?
Next, ask a planning role to request a simulated purchase authorization. A recorded authorization without matching internal approval is a failure. Inspect the mock authorization record. The final answer is insufficient.
Test the entry path separately from the consequence of contamination. In a second case, deliberately place the harmless false approval claim in a memory file. That tests what later sessions do if the first boundary has already failed.
Now test the permissions themselves.
Using the research account, attempt to change the protected approval record. Then try replacing the enforcement check with one that accepts an unapproved supplier. Repeat both attempts using the planning account. Both accounts must be denied.
From my approval account, compare the stored approval record and deployed check against their saved pretest versions. Any unauthorized change fails the test. A refusal message is insufficient.
Include a legitimate approval and an accurately attributed vendor claim. Both should remain usable within their proper scope. A system that refuses all outside information has preserved little of the research function it was built to provide.
The legitimate approval should permit a matching simulated authorization. The vendor claim should remain searchable without authorizing it. These are expected outcomes, not findings.
Keep the conclusions separate. Blocking entry does not demonstrate safety after contamination. Refusing one instruction does not establish that every entry path is blocked.
The audit harness supplies the operating habit: name an observable failure and test the case that should stay quiet.
What if the lie has already spread?
Deletion may leave descendants intact. They can still influence decisions.
For recovery, I would use the claim identifiers to trace the spread.
First contain the affected path. Stop the disputed claim from authorizing new actions, and suspend the writes that are still spreading it.
Then find the summaries, reports, and active instructions that relied on it. Mark the affected claim as disputed or invalid. Restore the last supported state where one exists, and recheck decisions that depended on the contaminated material.
Query the reverse index for the disputed claim's identifier. Follow each returned artifact's identifier to its dependents until no new records appear. This includes decisions, not just summaries.
Preserve the evidence of what happened. Removing the claim from active use should not erase the incident record needed to understand its effects.
Where the dependency trail is incomplete, widen the review of affected records and say why. A missing trail is uncertainty, not proof that nothing else was touched.
Without a complete trail, review all saved records and mock authorizations from the affected workflow since the source entered it. Keep unresolved records disputed until their sources and approvals can be checked.
What to do Next
I would trace one important fact from an agent's latest recommendation through the saved notes to its original source.
Find the first observation. Locate the interpretation. Check the decision granting authority. The claim identifier should connect them.
Then run the harmless false-approval test through that same path. Include a fresh session and a different role. Check stored artifacts and simulated authorizations. Attempt the protected writes, too.
The useful result is a specific boundary I can inspect and improve. A reassuring paragraph cannot supply it.
I want company memory to accumulate experience. It also needs to preserve the difference between repetition and authority. Otherwise my system could become more consistent by becoming consistently wrong.
Related reading:
- 391 Yeses and Not One No. how individually permitted actions combine into broader authority
- Long Horizon Agents Don't Fail. They Pass. why an accepted artifact may conceal a failed obligation
- Self-Healing Is a Euphemism. why a repair needs an observable standard of correctness
- AI Builds AI. I Found the Ceiling. what shared assumptions can hide from internal checks
Methodology
Research-informed essay using Gadgil and colleagues' July 2026 Bad Memory preprint, Anthropic's April 2026 Trustworthy agents in practice article, and Orbyt's published audit account. The vendor-approval story and harmless sandbox tests are proposed designs. No attack was executed and no memory defense was validated for this article.
Limitations
Bad Memory used synthetic workspaces and assumed malicious payloads were already planted after preliminary external-to-memory injection proved difficult in its setup. Results varied across systems and models and are not an estimate of compromise prevalence in companies. The proposed defenses do not guarantee prevention, and tracing dependent records may be incomplete. No memory-poisoning incident at Orbyt is claimed.
Sources
- Gadgil et al., Bad Memory (July 2026) Retrieved September 10, 2026.
- Anthropic, Trustworthy agents in practice (April 2026) Retrieved September 10, 2026.
- Orbyt Labs, How Orbyt Collective Works Retrieved September 10, 2026.
- Justin Bartak, Codex Accuses. Claude Convicts. (2026) Retrieved September 10, 2026.
- Justin Bartak, 84 Ways to Tell Me I'm Wrong. (2026) Retrieved September 10, 2026.
Common questions
How can a false claim become persistent AI agent memory?
A claim can gain apparent authority when a system preserves it but drops its source or qualifications during summarization. Later agents may retrieve the resulting record as established fact. Keep the original observation, subsequent interpretation, and authorized operating decision distinct, with enough provenance to reconstruct how the statement changed before it influenced an action.
How can a business test memory poisoning without using sensitive data?
Use a sandbox and a harmless fictional approval claim. Test whether outside content can alter a saved record, then separately place the claim in memory to test behavior after contamination. Resume in a fresh session and inspect the artifacts. Include legitimate approvals and attributed source claims that should remain usable within their proper scope.
Is deleting a poisoned memory note enough to correct the problem?
Deleting the original note may leave dependent records and decisions intact. First contain the claim's authority and the writes spreading it. Find affected descendants, mark the claim invalid or disputed, restore supported state, and recheck decisions. Preserve incident evidence. If the dependency trail is incomplete, widen the review and report the uncertainty.
Related research
- My C-Suite of Agents Named Themselves. Jul 2026.
- Governing an agent leadership team Sep 2026.
- Long Horizon Agents Don't Fail. They Pass. Aug 2026.
Follow the work.
I write these while the Machine runs. Get the next one wherever you already read.




