Giving the Site a Memory with Redis Agent Memory Server (Part 4)

July 15, 2026·8 min read

By the end of Part 3, the site had its own LLM running on the desktop under my desk, reachable through a Cloudflare Tunnel, with a feature flag to fall back to my friend's box. One problem remained: every conversation started from zero. You could tell the chat your name, your favorite database, your deepest fears about Ingress controllers, and thirty seconds later it had the memory of a goldfish with a busy schedule.

Then I found agent-memory-server, an open source memory service from the Redis team. This post is how I wired it into the site's chats, running entirely on my own hardware, and the two or three ways I got it wrong before I got it right. Fair warning up front: I am maybe a week into using this thing. Consider this a field report from early territory, written down while the mistakes are still fresh.

The Goal

Three things again, because that format keeps working:

1. Give the site's AI features durable memory. Tell the chat something once, phrase it as "remember that ...", and have it recalled days later, in a different conversation, with no login and no account.
2. Run the whole memory stack on the house box. Redis, the memory service, and every model call it makes should hit my Ollama, with no OpenAI key anywhere in the chain.
3. The real endgame: first watch how it learns about a visitor, then flip that around and use it to help the visitor learn faster. The k8s exam prep tool covers KCNA, CKA, and CKAD, and a coach that actually remembers which domains you keep bombing is a better coach than one that meets you fresh every visit.

Same house rules as the rest of the series: no keys in the browser, and nothing on my network exposed without a token.

The Building Blocks

agent-memory-server gives an AI agent two kinds of memory. Working memory is the conversation itself, per session: you PUT the messages, it holds them with a TTL. Long-term memory is the interesting part. A background process reads the working memory, uses an LLM to extract durable facts ("user prefers dark mode", "user lives in Memphis"), deduplicates them, embeds them, and indexes them for semantic search. You never write long-term memories by hand. You hand it conversations and it does the remembering.

Redis is the storage under all of it: the sessions, the extracted memories, and the vector index the semantic search runs on. I am running the redis-stack image in Docker next to the memory server.

Ollama, still the workhorse from Part 3. The memory server needs two models of its own: an embedding model for the vector search and a generation model for the fact extraction. Both point at the same Ollama that serves the chat, which means the entire memory system runs without a single external API call.

The whole stack is one Docker Compose file:

services:
  redis-memory:
    image: redis/redis-stack-server:latest
  agent-memory:
    image: redislabs/agent-memory-server:latest
    environment:
      REDIS_URL: redis://redis-memory:6379
      OLLAMA_API_BASE: http://host.docker.internal:11434
      GENERATION_MODEL: ollama/qwen2.5:3b
      EMBEDDING_MODEL: ollama/nomic-embed-text

The Architecture

Browser (Neon GPT, Station Intelligence, exam coach)
  │  POST /api/llm or /api/ask   { messages, memory: { userId, sessionId } }
  ▼
Cloudflare Pages Function
  │  recall:  POST /v1/memory/prompt      (before generating)
  │  store:   PUT  /v1/working-memory/…   (after answering, in the background)
  ▼
memory.chrishouse.io   (Cloudflare Tunnel + Access service token)
  ▼
agent-memory-server ──► Redis (sessions, facts, vectors)
        │
        └──► Ollama (extraction + embeddings, same box as the chat model)

The tunnel and Access setup is a straight copy of Part 3: a second hostname on the same tunnel, path-restricted to the API, behind its own service token. Anyone hitting memory.chrishouse.io without the token gets a 403 from Cloudflare's edge before the request ever reaches my house.

Identity is an anonymous id the browser mints once and keeps in localStorage. No accounts, no emails, no names unless you volunteer one to the chat. The memory is scoped to that id, so your remembered facts are yours and the next visitor gets their own clean slate.

Doing It Wrong First

My first integration treated the memory server like a database. Every chat turn, my code wrote its own long-term records: one episodic record of the exchange, plus a "durable fact" record whenever someone said "remember that ...". For recall, it ran four separate searches (semantic, keyword, hybrid, and a topic-filtered one) and stitched the results into the prompt by hand.

It worked, in the sense that a car with square wheels technically moves. What I actually built was a duplicate factory. The server was extracting memories from the conversations on its own, like it is designed to, while my code wrote overlapping records next to them. Within a day of testing, the store contained "chris likes pineapple pizza" four times, a server-merged memory that said pepperoni instead, and an episodic transcript of me asking about pizza. The site held contradictory pizza beliefs simultaneously, which is very human of it, and also useless.

The fix was deleting code. The idiomatic pattern is embarrassingly small, so here it is as pseudo code instead of a wall of JavaScript:

store(turn):
    PUT working-memory(sessionId, messages)
    # that's it. the server extracts, dedups, and indexes in the background

recall(question):
    POST memory/prompt(query, sessionId, semantic search over this user)
    # returns the session context plus relevant long-term memories,
    # already formatted to drop into the system prompt

One write, one read. Half the GPU work, no duplicate records, and the server's own deduplication actually gets a chance to do its job. If you take one thing from this post: do not hand-write long-term memories. The extraction pipeline is the product.

The Extraction Model Matters More Than I Expected

The memory server's extraction step is an LLM call, and my first configuration pointed it at the model I already had loaded: qwen2.5-coder, the code-tuned model that answers the chat. It extracted exactly zero memories. Not bad memories. Zero. A code model asked to distill "remember that my keyboard is a HHKB and I live in Memphis" into facts just stares at you and thinks about brackets.

Swapping to a general instruct model fixed extraction immediately, and then created a new problem: the 7B version plus the 14B chat model plus the embedding model added up to more VRAM than the card has, and Ollama started playing musical chairs with 16GB of memory. At one point the chat model got stuck mid-eviction and the whole endpoint hung. The 3B instruct model extracts nearly as well, fits alongside everything else, and the musical chairs stopped.

The lesson generalizes: extraction quality is a function of the model doing the extracting, and it is a separate decision from your chat model. Small general instruct beats large code-tuned at this job by an infinite margin, because anything beats zero.

What It Powers Today

"Remember that ..." in the chats. Tell Neon GPT or Station Intelligence to remember something, come back tomorrow in a fresh conversation, and ask about it. The recall goes through the same memory prompt as everything else, so facts surface even when you ask sideways ("what keyboard do I use?" finds a memory that never contained those exact words).

The station recognizes regulars. When Station Intelligence boots and memory exists for your browser id, the greeting is composed from what it knows. Mine mentions blue fish, for reasons I have fully earned.

The exam coach. This is the part I care about. Finish a quiz or practice exam on k8s exam prep and the page logs a summary to memory: score, domains, where you were weak. The coach button recalls that history and prescribes what to drill today. Next to it, a "drill weak spots" button builds your next quiz directly from the questions you have missed and your lowest-mastery domains, and those results feed back into memory. The loop closes: study, get remembered, get coached, drill the gap, repeat.

That third one is the experiment I wanted to run all along. The site learning my pizza order is a party trick. The site remembering that I cannot tell a NetworkPolicy from a hole in the ground, and quietly building tomorrow's study session around that fact, is the thing that might actually be worth the Redis container.

Early Days, Honestly

A week in, here is what is rough:

• Contradictions accumulate. Telling the chat "I no longer like pineapple pizza" adds a new memory next to the old one. The server has forget and compact endpoints built for exactly this, and I have not wired them up yet. Right now the model gets both facts and has to sort out the timeline itself.
• One enrichment step fails on my Redis. The topic/entity tagging task uses a command that only exists in Redis 8, and the redis-stack image I deployed predates it. Core memory works fine (store, search, recall are all healthy), but every extraction logs an error for the metadata step. The fix is moving to the redis:8 image, which now has vector search built in.
• Extraction is eventually consistent. Facts land seconds after the conversation, on a debounce, after a background model call. Say "remember X" and immediately ask about X in a new session and you can beat the pipeline. In practice nobody chats that way, but tests do.
• My own bug was the best one. After the refactor, every "remember that ..." reported a failed write, because my handler still checked a response field the old code returned and the new code did not. The writes were succeeding the whole time. The site was gaslighting itself.

None of these have made me regret the setup. All of them are the kind of thing you only learn by running it.

Design Decisions

Why memory lives behind the Pages Function. The browser never talks to the memory server. The function resolves the identity, loads memory into the prompt, and stores the turn afterward in the background, so a slow memory call never delays the answer. If the memory server is down, everything fails open and the chat just answers without memory. The site loses its memory gracefully, like the rest of us.

Why the identity is anonymous. An id in localStorage costs nothing, needs no consent screen beyond what analytics already covers, and gives exactly the scoping memory needs. Accounts would make the memory better bound and the site worse.

Why extraction runs on a small model. Covered above, but as a decision: the extraction model is infrastructure, and infrastructure should be the smallest thing that does the job, especially when it shares a GPU with the model people are actually talking to.

Why I am writing this now instead of after a month. Because the early mistakes are the useful content. A month from now the duplicate factory and the code-model-extracts-nothing discovery would be smoothed over into "configure it correctly", and you would step on both rakes yourself.

The Gotchas, So You Skip Them

• Do not write long-term memories by hand. PUT working memory and let the server extract. Hand-written records fight the deduplication and double your GPU load.
• Use a general instruct model for extraction. Code models extract nothing. Check your VRAM budget: chat model + extraction model + embeddings all resident at once.
• Session and user ids have a minimum length. Short test ids get silently replaced by fallbacks, and then your writes and reads disagree about identity. An hour of my life, gone.
• Check the response shape after upgrades. The memory prompt endpoint returns message content as objects, and my code assumed strings. Fail-open code hides that kind of bug until you go looking.
• Old Redis, new commands. If extraction logs unknown-command errors, your Redis predates what the memory server expects. Core features may still work, which makes it easy to miss.

Conclusion

The stack from Parts 1 through 3 gave the site a mouth. This part gives it a hippocampus, one Docker Compose file and a tunnel hostname away, running on the same GPU as everything else for the same monthly cost of zero dollars.

I am early on this, and the honest status is: durable memory works, recall is better than I expected, hygiene needs work, and the contradiction problem is real but survivable. The experiment that matters is the exam coach. If a site with memory can watch you study, notice what you get wrong, and shape what you practice next, that is a genuinely different thing from a chat widget with a good personality. Ask me in a month whether it made me faster at the CKA. The site will remember either way.

Tags:

Enjoyed this post? Give it a clap!