Part 3 of the Building the AI Memory Stack series
After finishing Part 2, I noticed something.
The browser tabs I had open while writing it were gone. The temporary notes were gone. The diagrams existed only while I was drafting.
The article remained.
That is the question underneath this entire post. Why did one thing survive when everything else disappeared?
A reader asked a version of it directly:
"If Active Working Memory is assembled for each task, where does all of that information come from?"
Most conversations stop at a simple answer.
"The vector database."
That answer isn't wrong.
It's just incomplete.
A vector database is one implementation of durable memory. It is not the architectural definition of durable memory.
Those are very different ideas.
Durable Memory Is Curated
If Active Working Memory is RAM, Durable Memory is not simply "disk."
Disk stores everything.
Durable Memory stores what the system intentionally decides to preserve.
Durable Memory is not a place. It's a policy.
That is a much narrower responsibility.
A durable memory layer may contain:
- Specifications
- User preferences
- Signed evidence
- Architecture Decision Records
- Policies
- Verified observations
- Structured domain knowledge
- Historical interactions
Notice what is missing.
- Scratch calculations
- Intermediate reasoning
- Temporary tool output
- Duplicate information
- Ephemeral context
Those things may have been useful.
That does not mean they deserve to survive.
What Survives Matters
Human memory works the same way.
You don't remember every sentence you read yesterday.
You remember what became worth remembering.
Agentic systems face exactly the same problem.
Not everything that passes through inference deserves to become memory.
Consider the kind of task from the last article: an agent maintaining an SDK. In a single pass it might retrieve several Architecture Decision Records, read a dozen Git commits, inspect a couple of open issues, call three tools, and generate intermediate summaries along the way.
When the task finishes, should all of that become memory?
Of course not.
Durable Memory is not everything the system observed. It is what the system intentionally decided was worth preserving.
Memory Is a Write Problem
One pattern I've noticed across many AI systems is that enormous effort goes into retrieval.
Teams debate embedding strategies, chunk sizes, hybrid search, semantic similarity, and re-ranking pipelines.
Yet comparatively little attention is paid to the opposite question.
Should this be remembered at all?
That is fundamentally a write-side decision.
Traditional software engineers already make this decision every day. We don't check temporary variables into Git. We don't commit compiler output. We don't version our cache directories. We deliberately preserve the artifacts that represent knowledge and discard the ones that existed only to complete today's work.
Durable Memory asks an agentic system to make the same distinction.
Every stored artifact becomes future context.
Every stored artifact has a maintenance cost.
Every stored artifact competes for future retrieval.
Every write is a promise to your future retrieval system.
Memory is not free simply because storage is inexpensive.
A system that remembers everything eventually remembers nothing particularly well.
The specification has a name for that failure state: the Digital Attic, where everything is kept and nothing can be found.
And when a Digital Attic gets queried, it hands your application a poisoned working set—a mix of current requirements, obsolete notes, and conflicting observations.
When that un-sieved context hits the context window, the system falls into Agentic Thrashing: spending precious inference cycles attempting to reconcile contradictory history rather than making forward progress.
The Difference Between Storage and Memory
This is why I think storage and memory should be treated as separate architectural concepts.
Storage answers:
Can we keep this?
Memory answers:
Should we keep this?
Those are different questions.
A filesystem stores.
A database stores.
An object store stores.
Durable Memory decides.
The Write Boundary
In traditional software architecture we spend a great deal of time discussing APIs.
In agentic systems, I increasingly think the more important boundary is the write boundary, what the specification calls Write-Side Custody.
Every piece of information attempting to cross into Durable Memory should answer questions such as:
- Is this authoritative?
- Is it verified?
- Does it duplicate existing knowledge?
- Does it expire?
- Can its provenance be established?
- Is it useful outside the current task?
Those questions determine whether something becomes memory or remains temporary context.
This Is Where Provenance Begins
This is also the point where the Sovereign Systems Specification begins to diverge from many AI architectures.
A memory that cannot explain why it exists is difficult to trust.
If an observation enters Durable Memory, the system should be able to answer:
- Who created it?
- When?
- Under what authority?
- Based on what evidence?
- Has it changed?
- Can it be verified?
Without those answers, Durable Memory slowly becomes institutional folklore rather than institutional knowledge.
Information without provenance is just gossip.
Durable Memory Is an Architectural Responsibility
Just as the previous article argued that Active Working Memory is more than prompt construction, Durable Memory is more than persistent storage.
It is memory as infrastructure: the architectural responsibility for deciding what knowledge deserves to outlive the task that created it.
That responsibility shapes every article that follows.
Deciding what deserves to survive is only the beginning.
The next question is whether the path that produced that knowledge can itself be examined.
That is where Part 4 begins.
Top comments (24)
The "Digital Attic" framing hits hard. I've seen this in robot learning systems — a trajectory passes every structural check, gets stored, gets used for training, and quietly poisons the policy. The cost isn't retrieval noise; it's corrupted gradient updates.
Same root cause: everyone obsesses over retrieval quality while the write side — should this exist in memory at all? — gets treated as trivially solvable because "storage is cheap."
Cheap storage just means you can afford a bigger attic.
honestly i start thinking vectorstore+sql is the right option.
the write boundary framing is the part i think most people are going to skip past, and it deserves the weight you put on it. memory is a write problem is the correct diagnosis.
one thing i ran into that i think sits just past your boundary. you ask "does it expire" at write time. i spent a while on the case where the answer at write time is no and it becomes yes later without the entry changing at all. i was testing against a live certificate authority and then a live mandate registry, and the shape that kept showing up was a stored fact that is still perfectly valid by its own ttl while the source underneath it revoked or narrowed the scope it was granted under. nothing about the entry is stale. the authority it inherited is. write side custody cant catch that one because the fact changes after the write, so it needs a partner at read time that re derives against the source instead of trusting the timestamp.
the other one i got wrong before i got it right. i assumed that if you curated properly the retrieval problem mostly went away, so i built a scorer that weighted governance signals and expected it to beat plain bm25. it didnt. that got falsified and i had to publish it. what survived was narrower and more useful, relevance and authority are different axes and retrieval ranks on the first one. so even a clean durable memory with no attic in it can hand back two entries that both passed your boundary honestly and still disagree, and nothing in the ranking knows which one governs.
curation fixes volume. it doesnt fix jurisdiction. that second gap is where i keep ending up.
looking forward to part 4, the examinability question is the right next one.
This is a really interesting distinction, and I think you're right that it sits just beyond what Write-Side Custody alone can guarantee.
Custody can establish that a fact was authoritative when it crossed the write boundary. It cannot guarantee that the authority remains unchanged forever. Your certificate and mandate examples make that especially clear: the stored fact hasn't necessarily become stale, but the authority that made it valid has changed.
That suggests a useful separation between write-time authority and read-time authority. Write-Side Custody establishes the former. Hydration or retrieval may need to revalidate the latter before allowing durable knowledge back into active reasoning. In other words, provenance can't always be treated as historical metadata. Sometimes it remains a live dependency.
I also really like your relevance-versus-authority distinction. A retrieval system can legitimately return the most relevant result while still returning the wrong governing result. Curation reduces the amount of noise in Durable Memory, but it doesn't resolve jurisdiction, supersession, or competing authority.
"Curation fixes volume. It doesn't fix jurisdiction." is a very useful way of putting it.
You've given me something to think about here, particularly around whether continuing authority belongs explicitly at the Hydration Boundary rather than being treated only as a property established at write time.
hydration boundary is the right place to put it, and separating write time authority from read time authority is a cleaner cut than the one i came in with. i want to push one step past it though, because i think the boundary moves one more time and thats the part that actually got me.
revalidating at hydration proves the authority was good at hydration. it doesnt prove it was good at use. a long running agent hydrates once and then reasons for forty minutes off what it pulled in. if the mandate gets narrowed at minute nine, the hydration check already passed, and it passed honestly. so youve tightened the window from authoritative at write to authoritative at read, which is a real gain, but its still a snapshot and the exposure is whatever span sits between the check and the act.
i walked into that exact shape somewhere else and it cost me. i had a live check come back clean and i read it as covering a period, when all it could ever prove was that instant. the fix wasnt a better check. it was noticing that a snapshot cannot make a claim about a span no matter how fresh it is.
i think thats also why ttl feels like it ought to cover revocation and never does. expiry is computable locally, its a prediction the entry carries about itself. revocation is news, it only exists at the source and the only way to have it is to ask. you can cache a prediction. you cant cache news.
which is why i dont think any of this shrinks the write side. i think it adds a required field to it. hydration can only revalidate what the write boundary bothered to record a handle for. if you stored the fact and the ttl but not who granted it and where you go to ask about it, then read time has nothing to re derive against and it quietly falls back to trusting the timestamp again, which is the thing you were trying to get away from. so write side custody still does the load bearing work here. it just has to carry the provenance handle forward instead of treating provenance as something it closed out at write.
the fork i dont have a clean answer to is what revalidation does when it cant reach the authority. fail open and youve built nothing. fail closed and the registry having a bad afternoon takes the agent down with it. i only have the two bad options on that one, so if part 4 goes near it im interested to see which way you go.
I think you've pushed this into an even more useful distinction: authoritative at write, authoritative at hydration, and authoritative at use are three different claims about three different moments in time.
The snapshot-versus-span observation is the key. A successful revalidation can prove something about the instant at which the check occurred. It can't magically extend that proof across the next forty minutes of agent activity. That means hydration can narrow the exposure window, but it can't eliminate it when authority is revocable during execution.
I also really like the TTL/revocation distinction. "You can cache a prediction. You can't cache news" gets at something important. Expiration can travel with the artifact because it is locally computable. Revocation remains dependent on an external authority because it represents a new event.
And yes, I think that strengthens rather than weakens Write-Side Custody. Provenance can't just record where a fact came from. For revocable authority, custody needs to preserve enough of a provenance handle that some later boundary can re-derive authority from the source. Otherwise we've preserved history, but not the ability to determine whether that history still governs.
The fail-open/fail-closed problem is where I don't think I'd want the architecture to prescribe one universal answer. My instinct is that inability to revalidate becomes a policy decision based on the consequence of the action. Reading a document and transferring money probably deserve very different failure semantics.
I'm also mulling over that hydration isn't necessarily the last authority check. For a long-running agent, there may need to be an execution-time check immediately before a consequential action. I'm going to think about where that responsibility belongs in the existing boundary model before giving it a new name, because I don't want to manufacture terminology where an existing boundary already owns the problem.
This is a really useful edge case. I may owe you a footnote by the time I'm done with this series.
write, hydration, use as three separate claims about three moments is the version i wish id had
when i started, that is cleaner than how i was carrying it.
the policy answer on fail open versus fail closed is right and i want to push on one thing inside
it, because i think it has a dependency that bites in exactly the place you put it.
if the failure semantics depend on the consequence of the action, the system has to know the
consequence class of the action at the moment revalidation fails. so where does that
classification come from. if it is looked up out of the same durable store whose authority you
just failed to re derive, you are using unvalidated state to decide how strictly to treat
unvalidated state. that is circular, and it fails in the worst direction, because an attacker who
can influence the record can mark a transfer as a read and the system will happily fail open on
it under a rule that was written to be careful.
i hit the same shape one level down and it cost me a published result. my gate took its
destination from the value sitting in the observer ledger, so there was no independent binding
between what a recovery actually read and what the ledger said it returned. the ledger was the
read source, not a witness to it. every verdict downstream inherited the assumption instead of
testing it. yours is the same defect wearing a policy costume.
the way out i think is your own distinction applied one layer up. consequence class has to be
locally computable, a property of the action being requested and the call site invoking it, not
something retrieved. transfer money is a fact about the tool being called. it should be as
computable as expiry, and for the same reason. authority is news so you have to ask. consequence
is a prediction so you must not have to.
on execution time checks, i think your instinct not to name a new boundary is right, and here is
the reason id give for it. adding a third checkpoint does not close the gap, it just makes the
span shorter. any pull based check, at any number of points, proves the instant it ran and leaves
whatever comes after. you cannot poll your way to zero. what actually closes it is direction
rather than position, the authority reaching in to invalidate rather than the agent reaching out
to ask. so its not a missing boundary, its an existing one that only runs one way.
and i should be straight that i do not have that built. i have the diagnosis and no push channel,
and a revocation that can interrupt in flight needs the agent to hold a durable identity the
authority can address, which is its own problem and not a small one. so i am handing you the
shape, not a solution.
I think "direction rather than position" is the piece that closes the loop for me here. We've been progressively shortening the interval by moving validation from write, to hydration, to use, but you're right that another pull-based checkpoint still only proves authority at another instant. It reduces the exposure window without changing the underlying temporal problem.
Your consequence-class point is equally important. If failure policy depends on whether an action is consequential, that classification can't inherit its authority from the state we're currently unable to validate. Otherwise the fallback mechanism contains exactly the circular dependency it is supposed to protect against. Making consequence class locally computable from the requested capability and call site gives it a much cleaner trust root.
That also suggests two different categories of information traveling through the system. Some properties can be carried and evaluated locally, such as expiry and consequence class. Others represent changing external reality, such as revocation, and therefore require an authoritative source. Your distinction between prediction and news is proving surprisingly useful here.
And I agree that push-based invalidation changes the problem rather than merely moving the checkpoint. It introduces some difficult questions around durable agent identity, addressability, interrupted execution, and what happens when the revocation channel itself is unavailable, but those feel like the right problems to be asking rather than inventing another validation boundary.
I'm going to capture this thread and think about it against the existing Sovereign boundary model before deciding whether anything needs refinement. You've given me a considerably better formulation of the problem than the one I started with. Thanks for pushing on it.
you flagged what happens when the revocation channel itself is unavailable as one of the hard
ones, and i think thats the strongest argument for push rather than a problem with it.
a channel that is down can be observed to be down. thats a detectable state, you can alarm on it,
you can degrade deliberately, you can decide in advance what silence from the authority means.
compare that to what we have now, where a fact whose authority quietly changed underneath it is
not observable from the holder's side at all. nothing to detect, nothing to alarm on, no signal
of any kind.
so push does not make the failure go away. it converts an invisible failure into a visible one.
that is not a solution and i wouldnt sell it as one, but it is a strictly better place to be
standing, because you can only build policy on top of something you can see.
good thread. you took it further than i had it.
Memory is far from just storage, it's also the routes to access that storage. Insights, recalled similarities etc. You can't model memory like a computer alone.
I really like the distinction between storage and memory, especially the idea that every write is a promise to the future retrieval system.
One thing I’d add is that provenance alone may not be enough for project memory. A decision can have perfectly valid provenance and still become misleading later because the constraints or evidence behind it changed.
For example, an architecture decision may be authoritative, verified, and correctly attributed — but six months later the dependency, scale requirement, cost constraint, or API it was based on may have changed.
That makes me think durable memory needs two related histories: what was known/decided and what made it trustworthy at the time.
The harder problem then becomes detecting when the second one has changed, so the first one gets questioned rather than blindly retrieved.
Curious if your write-side custody model is thinking about this kind of dependency-driven invalidation, or if that sits outside the durable-memory boundary?
I think you're pointing at an important distinction between provenance and continuing authority. Provenance can tell us why an ADR was trustworthy when it was written, but it can't guarantee that the dependency, constraint, API, or policy that made the decision valid hasn't changed since then.
I've been thinking about that as three separate claims: authoritative at write, authoritative at hydration, and authoritative at use. Write-Side Custody can establish the first, but the other two may require revalidation against the original authority or dependencies rather than simply trusting what was recorded historically.
Your idea of preserving both the decision and what made it trustworthy at the time fits nicely with that. I think the write boundary needs to preserve enough of that dependency/provenance information that a later system can question the decision when its supporting conditions change.
So I don't think dependency-driven invalidation sits outside Durable Memory so much as exposes a limit of what the write boundary alone can guarantee. A correct write can still become an incorrect thing to use later.
Ran into exactly this building an entity graph last year. Retrieval was the part you could actually benchmark. The write side was where things quietly broke: conflicting source assertions, staleness never properly encoded, every high-confidence link that eventually aged into wrong. We started calling our Attic version "confident garbage", indexed and retrievable but stale by the time anything actually queried it.
"Confident garbage" is a fantastic description of the failure mode. The dangerous part isn't that the information becomes impossible to retrieve. It's almost the opposite: stale information remains perfectly indexed, highly retrievable, and potentially very convincing.
That's a big part of what I mean by the Digital Attic. The problem isn't simply accumulating too much information. It's accumulating information whose authority, provenance, or temporal validity becomes unclear, while leaving it fully available for future reasoning.
Your entity graph example also reinforces why I keep coming back to the write side. If you don't model staleness, source authority, supersession, and conflicting assertions when knowledge becomes durable, retrieval eventually inherits a problem it isn't equipped to solve.
The write-boundary framing is the key reframe here. Most memory
discussions I've seen treat retrieval quality as the whole problem,
this makes the case that a bad write policy poisons retrieval no
matter how good the embedding/ranking is downstream, since you're just
ranking garbage more efficiently.
The provenance requirement is what separates this from "just add a
vector DB" the most. "Who created it, based on what evidence, can it
be verified" turns memory into something with accountability instead
of a growing pile of past outputs the system trusts equally regardless
of source.
Curious how this handles information that was true when written but
has since become stale, not wrong exactly, just outdated. Does the
write boundary include an expiry/re-verification policy, or does that
get treated as a separate concern from the initial "should this be
remembered" decision?
I think that's an important distinction, and I've actually been refining my thinking on it through some of the discussion around this post.
I'm increasingly thinking about authority as a temporal property. Write-Side Custody can establish that something was authoritative when it crossed the write boundary, but that doesn't guarantee it remains authoritative later. A policy can be superseded, an API can change, a dependency can disappear, or an external authority can revoke something without changing the stored artifact itself.
So yes, I'd want the write boundary to capture things like expiry and re-verification requirements, but also enough provenance to enable later revalidation. A TTL can tell us when something is expected to expire. It can't tell us that something changed unexpectedly outside the system.
That leads to a distinction I find useful: authoritative at write time and authoritative at use time are separate claims. The write boundary establishes the first. Later retrieval or Context Hydration may need to revalidate the second before allowing that knowledge back into active reasoning.
And importantly, outdated doesn't necessarily mean worthless. An old ADR or policy may remain valuable historical memory even after it stops being authoritative. I'd rather preserve that distinction than simply delete everything that becomes stale. Essentially, preserve the concept of "this part of our history, but it no longer governs" and move beyond a strict
valid = true/falsedistinction.The write-side framing matches what I've seen. Our rule ended up being: default is discard, persistence has to be earned. Anything the agent wants to keep goes through an explicit promotion step with a reason attached, basically an ADR for memory. It slows things down a little but it's the only way we've found to keep the store from filling up with plausible-looking intermediate junk.
One thing I'm still unsure about: how do you handle memories that were correct when written but went stale? TTLs feel wrong for durable memory, but never expiring anything has its own cost.
"Persistence has to be earned" is very close to how I think about it. I like the ADR-for-memory analogy too, particularly attaching a reason to the promotion rather than treating persistence as the default outcome of an agent run.
On staleness, I'm increasingly convinced TTLs only solve one category of the problem. Some knowledge genuinely has an expected lifetime, and a TTL works well there. But other knowledge becomes invalid because something external changed: a policy was superseded, an API changed, an authority revoked something, or the assumptions behind an ADR no longer hold.
In those cases, the memory itself may not have "expired" at all. The world around it changed.
That suggests Durable Memory needs more than expiration metadata. It needs enough provenance and dependency information that later retrieval or hydration can determine whether the knowledge is still authoritative before putting it back into active reasoning. So I wouldn't automatically delete old durable memory. Sometimes the fact that something used to be true is itself valuable. I'd rather distinguish historical validity from current authority.
One thing this thread treats as a read-time problem worked better for us as a write. When a decision stops holding, someone on the team knows first, usually the one who killed it. So in kgai we record the invalidation itself as an event, with a reason and a link to what replaces it. The dead entry stays in history, recall just stops returning it. No TTL guessing, no re-verification pass, the write boundary already told the store. And events have authors, so teammates who share the store get the correction instead of each rediscovering it. External authorities like the certificate case in the comments still need asking, team decisions mostly dont.
I like this distinction, particularly between internal decisions and external authorities. Treating invalidation as a newly authored event rather than mutating or deleting the original memory fits nicely with how I've been thinking about the write boundary.
The original entry remains historically true: we made this decision under these conditions, at this time. The invalidation is another fact: this decision stopped governing, for this reason, and this is what replaced it. Recall can project the current state without rewriting the history that produced it.
I think your last point is the important boundary. For team knowledge, someone often knows that the world changed and can write that change into memory. For an external authority, the memory system may never receive that event, so revalidation still has a role. The difference is whether authority changes enter the system as writes or must be discovered from outside it.
I also like the authorship piece. An invalidation with provenance is much stronger than silently marking something stale.
'every write is a promise to your future retrieval system' is the line that should be in every RAG architecture review. we spent six months tuning retrieval before someone asked what we were actually storing — a third of the vectors were intermediate reasoning traces from prior agent runs, never referenced again, actively poisoning cosine similarity scores.
the write side decision is where most teams lose. 'storage is cheap' doesn't mean writes are free when every one competes for attention at retrieval time.
do you find 'should this be remembered' easier with explicit schema types (ADR vs preference vs observation), or does it still need per item judgment?
That intermediate-reasoning example is exactly the kind of failure mode I had in mind. The individual writes look inexpensive, but collectively they change the retrieval environment for everything that follows.
I think explicit schema types help enormously, but I wouldn't let the schema make the final decision. An ADR, preference, observation, policy, or verified fact can each have different default retention rules, validation requirements, provenance requirements, and supersession behavior. That gives Write-Side Custody a much stronger starting point than evaluating an untyped blob.
But membership in a durable type doesn't automatically make a particular item worth preserving. An observation can still be duplicate, low-confidence, task-specific, or superseded. A preference can be temporary. Even an ADR can be replaced.
So I'd probably frame it as: schema determines the rules of custody; the individual artifact still has to earn persistence under those rules.
That also gives you much better behavior later because retrieval isn't merely searching "memory." It knows what kind of knowledge it retrieved and what authority that type is supposed to carry.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.