Why markdown rot breaks agent memory

The file that worked

The first version is always good. Someone notices the agent keeps reaching for the wrong test command, writes four lines into a CLAUDE.md, and the problem goes away. That is a genuinely excellent return on four lines. Zero infrastructure, works in any harness that can open a file, reviewable in a pull request, and it lives next to the code it describes.

So it grows. Someone adds the deploy runbook. Someone adds the reason the auth middleware is ordered the way it is. Someone adds a warning about the migration that has to run before the deploy. A year later there is a CLAUDE.md, an AGENTS.md that mostly duplicates it, a docs/ folder with fourteen files, and three READMEs in subdirectories that nobody has opened since they were written.

None of that is a discipline failure. It is what happens when a format with no lifecycle is used for facts that have one. The pile does not go wrong because it is markdown. It goes wrong in three specific ways, and each one has a specific shape.

Failure one: nothing on the page knows it went stale

Here is the sequence, and it is always the same sequence. In March you write down that background jobs go through the queue table, never through the worker directly. In June you rip out the queue table and move to a hosted broker, because the table was a bottleneck. The code changes. The tests change. The docs/jobs.md file does not change, because changing it was not part of the diff that made the change work.

In September an agent picks up a ticket about a failing job, reads docs/jobs.md, and writes a patch that inserts into a table that no longer exists. You catch it in review. Or you do not, and you catch it in production.

A file cannot tell you it has gone wrong. That is not a bug in markdown. It is the absence of a feature markdown never claimed.

The instinct is to blame the person who did not update the doc, but that is the wrong lesson. Nobody updates the doc, at any company, ever, reliably. What is missing is not diligence. It is a mechanism that distinguishes a fact somebody confirmed last week from a fact somebody wrote eighteen months ago and has not looked at since. In a folder of markdown files, those two facts are typographically identical.

Git timestamps do not close this gap, and it is worth being precise about why. A file's modification time tells you when its bytes last changed. It does not tell you whether anyone checked the claim. Reflow a paragraph and the file looks freshly maintained. Meanwhile a fact that has been correct and untouched for two years looks abandoned. The signal you want and the signal you have are close enough to be confused and far enough apart to mislead.

Failure two: retrieval by filename

The second failure is quieter, and it costs you before you notice it. To find a note in a folder, you have to already know roughly where it is, or guess the word it was written with.

Someone recorded that the rate limiter has to stay in front of the auth check, because moving it behind auth once let unauthenticated traffic burn the token budget. They wrote it in docs/incidents/2025-11-token-burn.md. Eight months later an agent is refactoring middleware order and searches for middleware, ordering, rate limit. The file says throttle and quota. No match. The agent moves the rate limiter behind auth, exactly as someone did the first time, for exactly the same sensible-looking reason.

This is what people mean when they say grep is not memory. Grep answers where does this string appear. Memory has to answer what do I need to know before I touch this, and the second question cannot be reduced to the first, because the person who wrote the answer and the person asking the question do not share a vocabulary. They are frequently not even the same species: one is a human writing at 11pm after an incident, the other is a model tokenizing a diff.

A search subagent helps here, and it is a real improvement over nothing. But it improves navigation of a library organised by filename. It does not make the library addressable by meaning, and it cannot surface a fact that shares no words with the thing you are doing.

Failure three: all or nothing loading

When the agent cannot locate the right paragraph, it does the only thing left: it loads whole files. A 400-line runbook enters the context window so that 6 lines of it can be relevant.

The cost is not only tokens, although it is also tokens. It is that the signal-to-noise ratio of the context window is the single biggest lever on whether the model does the right thing, and stuffing it with mostly-irrelevant prose moves that lever the wrong way. You are paying to make the answer worse.

The alternative most teams land on is to put the important things in the always-loaded instructions file, which works until the always-loaded instructions file is 600 lines long and the model is skimming it the same way you would.

What a lifecycle actually requires

The interesting question is not where should notes live. It is what has to be true of a fact for it to be safe to serve to an agent. Working through that gives you a short list, and the list is the same whether you build it yourself or use something off the shelf.

Two clocks, not one

A fact needs two independent timestamps, and conflating them is the root of most of the trouble above.

• Freshness is when the content last changed. Cheap, automatic, and nearly worthless on its own.
• Verification is when somebody last confirmed the claim still holds against reality. Expensive, and it is the one that matters.

The important design rule is that an ordinary edit must never set the verification clock. If fixing a typo counts as verification, the signal is gone. In Stele, verification is only ever set by a deliberate check, which is what makes it possible to ask a question a folder cannot answer: which of our facts has nobody confirmed in ninety days?

Retirement chosen when the fact is written

Most facts are not permanent, and their authors usually know it at the moment of writing. The workaround we hit on is to make the writer decide the fact's ending at the same time they decide its beginning. There are three ways a fact can be scheduled to retire:

• Bound to a task. A risk like the migration has to run before the deploy or sessions drop is true until the migration ships. Bind it to that task and it retires itself when the task closes.
• Expiry date. The staging database is on the old schema until the end of the quarter is a fact with a known shelf life. Give it one.
• Nothing. Durable architecture and decisions do not expire on a clock. They retire by being replaced, which is a different mechanism.

The point of forcing the choice at write time is that cleanup stops being a chore somebody has to remember. Nobody schedules a quarterly audit of their notes folder. Everybody can answer when does this stop being true in the moment they are writing it down.

Replacement, not deletion

When a decision reverses, the wrong move is to delete the old one. You lose the answer to why did we change our minds, which is usually the most valuable thing in the file. The right move is to record that the new decision supersedes the old one, keep the old one readable, mark it clearly, and rank it below its replacement so it only surfaces when somebody is genuinely asking about history.

That has a second benefit that took us a while to appreciate: a supersede recorded by mistake becomes a wrong label you can correct, instead of a true fact you destroyed. We wrote about that one separately, because the failure mode is more interesting than it sounds.

Status governs rank, never existence

A rule we got wrong and had to fix: for a long time, finished work was hidden from recall entirely, on the theory that a closed task is exhaust. Then we hit a case where a task had been completed four hours earlier, its completion note was the only written explanation of how the feature worked, and recall could not see it at all. Exact search found it instantly.

The model was missing time. A task finished this morning is the changelog. A task finished eight months ago is exhaust. Status alone cannot tell them apart, so status should never have been deciding visibility. Now it decides ranking, on a decay curve, and everything stays reachable with an honest label on it.

What that looks like day to day

Two of these run without anyone asking. Expiry and task-bound retirement fire on their own, and knowledge that has gone a long time without verification gets flagged rather than served as if it were certain.

The deliberate half rides along with the agents you already have. A verification sweep diffs against the exact commit it last reviewed, gathers the facts touched by what changed, and checks each one against the current code, both the claim and the files it points at:

The health check is the broader audit: overdue verification, expired notes, decisions superseded but still marked active, near-duplicates, edges pointing at nothing. What is safe to retire on its own is cleared in the same pass, and the rest comes back as a worklist.

Neither of these is magic. They are ordinary maintenance, with the difference that they are run by the agents doing the work rather than by a human who has to remember.

Anchors, or how a fact finds you

The retrieval failure earlier deserves one more note, because there is a fix for it that is easy to miss.

Text matching gets you a long way, but it fails exactly where the vocabulary diverges, which is exactly where you need it most. So a fact can also declare the files and symbols it is about. The rate limiter lesson does not have to share words with your refactor. It has to point at the file:

lesson: "Rate limiter must stay in front of the auth check"
  anchor_files:   [src/middleware/index.ts, src/middleware/rate_limit.ts]
  anchor_symbols: [buildMiddlewareChain, RATE_LIMIT_WINDOW]
  body: |
    Moving the limiter behind auth in Nov 2025 let unauthenticated
    traffic through to the token-metered path. 4h outage, $2.1k burn.
    The limiter has to reject before anything that costs money runs.

Now the trigger is not a keyword. It is the edit. Touch buildMiddlewareChain and the lesson arrives, whatever words you were thinking in.

This is also the honest answer to why not just use embeddings. Semantic search is a fuzzy match and it is genuinely good. Anchors are an exact one. You want both, because they fail in different places: a fuzzy match catches the fact you described in other words, and an exact anchor catches the fact you were not describing at all.

When the file is still the right answer

It would be dishonest to end without this part.

In-repo markdown needs no hosted backend, no account, and no network. It survives your vendor going away. It is reviewable in the same pull request as the code, which is a real advantage that nothing else on this page replicates. If you are one person on one project with a folder you can hold in your head, keep the folder. You will not get a return on anything more.

The trade tips when some combination of these is true:

• More than one person, or more than one agent, needs the same facts, and they are currently getting different ones.
• You have been bitten at least once by a reversed decision that an agent kept following.
• You cannot answer which of these notes is still true without reading all of them.
• The notes matter outside the checkout: in planning, in review, from a phone, from an agent that is not a coding agent.

And one boundary worth stating plainly, because it is the kind of thing people assume: the automatic parts run on their own, but the sweeps run when your agent runs them. A fully autonomous background gardener that tends the record with nobody prompting it is something we are building, not something that ships today.

If you take one thing from this: the problem was never the file format. It was that facts have a lifecycle and files do not. Any solution that does not model the lifecycle, including a more organised folder, is going to fail in the same three ways on a longer timeline.