Retrieval-augmented generation and agent memory are often discussed as competing architectures. They are not. Retrieval is a mechanism for finding useful context; memory is a system for maintaining useful state. A capable agent will often use retrieval inside its memory workflow, but a vector index by itself does not provide the guarantees people expect when they hear “the agent remembers.”
The distinction is easiest to see through a question: “What should the agent believe now?” RAG can return documents that mention the subject. It does not inherently determine which statement is current, whether two mentions refer to the same person, or whether the agent is authorized to use a particular fact for the present task.
What RAG does well
Conventional RAG is excellent when the job is to find passages relevant to a query. A typical pipeline chunks documents, creates embeddings, retrieves semantically similar chunks and asks a model to answer using them.
This works well for:
- locating policy language in a large corpus;
- answering questions from product documentation;
- finding prior tickets similar to a current issue;
- discovering evidence that uses different vocabulary from the query;
- narrowing a large archive before deeper reasoning.
RAG also has attractive operational properties. It can be added without redesigning the source systems, scales to large text collections and keeps the model’s immediate context smaller than the full corpus.
Nothing about people-aware memory eliminates those strengths. The mistake is asking similarity retrieval to perform jobs it was not designed to perform.
What durable memory adds
A memory system maintains entities and claims through change. It can say that a person currently holds one role, previously held another, made a commitment last week, corrected a mistaken preference yesterday, and belongs to a particular authority scope.
That requires operations beyond nearest-neighbor search:
- resolve and sometimes split identities;
- promote selected events into durable claims;
- attach effective time and expiry;
- supersede old state without deleting history;
- preserve provenance and epistemic status;
- reconcile contradictions;
- apply owner, tenant and task scope;
- accept explicit correction.
These are state-management problems. Embeddings may help generate candidates, but the system still needs rules and data structures that make the result inspectable.
The product model describes this as people-aware memory: durable relationship context prepared for a particular agent and task.
Similarity is not current truth
Consider an account with fifty documents that repeat the original success metric and one recent executive message that changes it. Similarity and frequency will favor the old metric. A memory system should favor the new authoritative statement while preserving the historical one.
The relevant operation is supersession:
- old claim: “Seat activation is the renewal metric,” effective January through April;
- new claim: “Weekly adoption is the renewal metric,” effective from 18 April;
- source: executive sponsor message;
- status: current;
- historical link: supersedes the January claim.
RAG can retrieve both source passages. It does not naturally produce the temporal relation between them. You can prompt a model to infer it on every query, but that repeats work, increases variance and makes correction difficult. Maintaining the relation once is usually safer and cheaper than rediscovering it indefinitely.
Chunk identity is not person identity
RAG systems tend to identify chunks by document and position. People memory needs canonical subjects that survive across sources.
A CRM contact, email sender and calendar attendee may be the same person. Two support users with the same name may not be. Embedding similarity can suggest a match, but identity resolution should use stronger evidence: verified addresses, organization, role, explicit links, source IDs and reviewable confidence.
This matters because retrieval quality cannot compensate for a contaminated entity. Once two people are merged, every “person-level” query can return a plausible mixture. Conservative identity is therefore a prerequisite to high-quality relationship retrieval, not a post-processing detail.
Memory writes need policy
RAG is usually read-oriented: ingest source text and retrieve it later. Agent memory also has writes. The system must decide what becomes durable.
A naïve memory agent may write every model-generated conclusion back into its own store. That creates a feedback loop: speculation becomes stored context, stored context is retrieved as evidence, and repeated retrieval makes the speculation appear confirmed.
A robust write path distinguishes:
- source events from model outputs;
- explicit user direction from extracted fact;
- direct evidence from inference;
- transient task state from durable relationship state;
- proposed memory from published memory.
Useful controls include promotion thresholds, review for high-impact writes, source requirements, expiry defaults and a prohibition on using a generated summary as sole support for a stronger claim. The memory-quality page outlines the role of evidence and correction in this boundary.
Retrieval still belongs inside memory
Once durable memory exists, retrieval remains essential. It can operate over several collections:
Raw evidence retrieval. Find source passages that support or challenge a claim.
Entity retrieval. Find the correct person or organization from partial cues.
Memory retrieval. Select maintained claims, commitments and relationships relevant to the task.
Change retrieval. Find events since the last briefing or since a claim was established.
Counter-evidence retrieval. Search for material that contradicts a proposed update before publishing it.
The architecture is not “database instead of RAG.” It is a layered system in which semantic retrieval, structured filtering and temporal state each handle the part they are good at.
A reference query flow
Suppose an agent receives: “Prepare me for tomorrow’s call with Maya about renewal risk.”
A mature flow might:
- Resolve “Maya” within the caller’s authorized account scope.
- Load the current person and account state: role, priorities, open commitments and known risks.
- Retrieve recent changes since the last substantive interaction.
- Search raw evidence for support and counter-evidence on the highest-impact claims.
- Identify contradictions or uncertainty that should be surfaced rather than hidden.
- Render a compact briefing tailored to renewal preparation.
- Include source references for consequential statements.
A generic vector query can contribute to steps three and four. It does not supply the complete flow.
When plain RAG is enough
Not every product needs durable memory. Plain RAG may be sufficient when:
- questions are primarily about documents rather than changing people;
- source authority and time are simple;
- each request is independent;
- a wrong answer can be cheaply verified;
- there are no durable commitments or corrections;
- the system does not make or prepare consequential actions.
For example, an internal documentation assistant can often rely on retrieval plus citations. If the corpus has versioned policies, metadata filters may handle freshness adequately.
The need for memory increases when the system spans repeated interactions, multiple sources, changing identities and decisions that depend on continuity.
When a state model becomes necessary
Add maintained memory when you encounter symptoms such as:
- the same person appears as several disconnected contacts;
- old facts outrank new ones because they are repeated more often;
- commitments disappear inside meeting summaries;
- users repeatedly correct the same mistaken conclusion;
- the agent cannot explain why it believes a relationship claim;
- personal and organizational context leak across task boundaries;
- briefings contain lots of relevant text but no clear “what changed.”
These are not prompt problems. They indicate missing state semantics.
Evaluation should separate retrieval and memory
If the architecture has both layers, evaluate them separately.
For retrieval, measure whether supporting evidence is found: recall, ranking quality, citation correctness and latency.
For memory, measure whether maintained state is right: identity precision, current-vs-superseded accuracy, commitment lifecycle accuracy, contradiction handling, correction propagation and scope enforcement.
Then test the end-to-end decision output. A briefing can fail because retrieval missed evidence, because memory promoted the wrong claim, or because rendering omitted an important uncertainty. Combining all failures into one answer-quality score makes the system harder to improve.
A sensible build order
A practical sequence is:
- Keep your existing source ingestion and retrieval.
- Introduce canonical entities with conservative matching.
- Add typed durable objects for one workflow, usually commitments or current priorities.
- Record provenance, effective time and epistemic status on every promoted object.
- Add contradiction checks and correction operations.
- Retrieve maintained state first, then use RAG to gather supporting and recent evidence.
- Render for a named task and authority scope.
This approach preserves the value of the existing RAG stack while adding the state semantics it cannot provide alone. For a more direct side-by-side, see compare approaches. For implementation boundaries beneath an existing agent, see developers.
RAG helps an agent find what was written. Memory helps it carry forward what remains true, what changed and what still needs to happen. The strongest systems do both deliberately.