How We Built Persistent Memory for an AI Companion

Ayra remembering your sister's name a month after you mentioned her is not one system — it is an extraction pipeline, a vector database, retrieval logic, deduplication rules, and a small set of hard-won failure modes. This is the actual architecture, from the team that maintains it.

Why chat history is not memory

The naive approach to companion memory is to keep the transcript and feed it back in. It fails three ways:

• →Context limits — a year of daily conversation can run to millions of words; no context window holds it, and stuffing it would be ruinously expensive per message.
• →Signal-to-noise — 90% of any transcript is small talk. Retrieval over raw transcripts drowns the signal ("my sister's name is Priya") in noise.
• →Structure — facts have types and relationships (people, preferences, emotional events). Transcripts don't, so every downstream system has to re-parse them.

The architecture

Extraction runs asynchronously after conversation activity — the user never waits for it, and a slow extraction job never delays a reply. Each extracted fact is stored twice, in effect: as a structured, human-readable entry (what you see in the memory manager in the app) and as a vector embedding (what retrieval searches).

The vector store we use is Qdrant; embeddings come from a Gemini embedding model; the structured store is PostgreSQL. Retrieval is scoped per-user by design — there is no path by which one account's memories surface in another's sessions.

Retrieval: relevance, not recency

At session time, what you say (or your opening greeting) is embedded and compared against your memory vectors. The most semantically relevant entries are selected — not the most recent. That's why asking about your exam pulls up the exam conversation from two weeks ago, even if you talked about a movie yesterday.

One subtle product decision: not everything retrieved is injected. The prompt builder assembles a compact memory context — a summary view, not a dump — because context quality beats context quantity. A model buried in 200 loose facts performs worse than one given the 12 relevant ones.

Deduplication and correction

If you mention your sister three times across a month, that should be one memory — updated, not tripled. The extraction step merges new facts into existing entries (a new job replaces the old one; a nickname is added to a person, not a new person). This is unglamorous work, and it is where most of the perceived quality of memory actually lives.

Correction is the sibling problem. Users fix Ayra in conversation ("no, her name is spelled Priyanka"), and the system is designed to treat those corrections as extraction input — the corrected fact replaces the wrong one. It is not perfect; sometimes a correction needs to be repeated. We'd rather ship honest memory with visible controls than memory that pretends to be infallible.

What can go wrong (our actual failure modes)

• →Wrong-fact persistence — if extraction misreads a joke as a fact, it can stick until the user deletes it. The visible memory manager exists precisely because we decided users must be able to see what's stored.
• →Over-retrieval — early on, too many loosely-related memories made Ayra bring up irrelevant things unprompted. Tightening relevance thresholds was a big quality jump.
• →Stale emotional state — "user was stressed in June" should not color every future session. Emotional memories need decay or context framing.
• →Extraction lag — because extraction is async, a fact told at the end of a session may not be available until the next one starts. We accept this tradeoff for latency.

Privacy: what the design guarantees

Memory is the most personal data a companion product holds, so the guarantees are architectural rather than promised:

• →Per-user isolation — memories are keyed to your account; retrieval filters to your data only.
• →User control — view, edit, or delete individual memories in the app; a full memory reset wipes the vector store entries alongside the structured facts.
• →No cross-user training — your conversations are not used to train Ayra's models for other users.
• →Deletion means deletion — forgotten entries are filtered out of retrieval immediately, not just hidden.

For the user-side view of all this — what companion memory means for you and your data across the industry — read AI Companion Memory and Privacy: What Users Should Know.

What we learned, condensed

• →Extraction quality > retrieval sophistication. Bad inputs can't be retrieved well.
• →Show users their memories. It converts "creepy black box" into "feature I control" — and it surfaces extraction bugs within days.
• →Fewer, better-injected memories beat many. Curation is part of the system.
• →Corrections are first-class facts, not edge cases.
• →Memory is a UX surface as much as a database. Ayra mentioning something at the wrong moment can feel worse than forgetting.