Presenters

Source

AI Agents Need to Remember: Designing Powerful Memory Systems with MongoDB 🧠💾

Ever wonder why AI assistants sometimes forget what you just told them, or give you wildly different answers even when you ask the same thing? It’s a common problem, and it all boils down to memory. In the rapidly evolving world of AI, especially with the rise of AI agents, building effective memory systems isn’t just a nice-to-have; it’s a must-have for delivering consistent, valuable results.

This workshop, held at dot.local London 2026, dives deep into the crucial topic of designing memory systems for AI agents. We’ll move beyond the stateless nature of current AI models and explore how to transform them into stateful powerhouses capable of remembering and learning.

The AI Memory Crisis: Why Current Systems Fall Short 📉

Let’s start with some eye-opening statistics. A survey by MIT revealed that a staggering 95% of organizations are seeing zero returns from their AI projects. This isn’t due to a lack of infrastructure, regulation, or talent, but rather a fundamental challenge: learning.

Generative AI systems are inherently stateless. They lose track of goals and previous details, leading to inconsistent answers and a lot of wasted effort. Simply upgrading to a better model won’t fix this. What these systems truly need is memory, much like humans possess.

What is an AI Agent? 🤔

Before we talk about memory, let’s quickly define an AI agent. Think of it as a system comprising:

  • An LLM (Large Language Model): The “brain” of the agent, like Gemini, Amazon Bedrock, or Azure’s AI Foundry.
  • Tools: These are APIs that the LLM can utilize. They can be as simple as a weather forecast API or as complex as specialized machine learning models.
  • Orchestration Code: This code connects the LLM with its tools.

When an input arrives, the LLM processes it, decides on the best action (which is usually calling a tool), executes that action, and feeds the outcome back to itself. This cycle continues until the agent has solved the problem.

The Problem with Statelessness: A Coffee Example ☕

Imagine you’re using a coding agent and you tell it, “My preferred database is MongoDB.” It responds positively. Later, you ask it to recommend a database for a new project. Without memory, it might suggest PostgreSQL or Oracle, completely forgetting your previous preference. This happens because LLMs only see what’s explicitly passed in each call; they don’t inherently remember past interactions. To make them remember, we need to help them persist information externally.

The Challenge of Infinite Memory: Token Limits 🤯

A naive approach might be to simply accumulate all previous information and pass it to every LLM call. However, LLMs have finite context windows – a limited number of tokens they can process at once. For agents, information accumulates rapidly through tool calls and reasoning traces. Even LLMs with large token limits (like 1 million) only delay the problem.

The Solution: External Memory Systems with MongoDB 💡

The genius solution is to design systems that allow AI agents to offload information to an external data source, like MongoDB, and retrieve it only when needed. This is where memory systems come into play.

But do we store everything? Not necessarily. Just like humans, not every detail is worth remembering. This leads us to the different types of AI memory, modeled after human cognition.

Types of AI Memory: A Human-Inspired Approach 🧠

We can broadly categorize AI memory into two main types:

1. Short-Term Memory: Session-Specific Recall ⏱️

  • Human Analogy: Remembering a recent conversation with a friend about a keynote.
  • For Agents: This is session history. It includes user messages, AI responses, and tool calls within a particular session. It’s about persisting insights from within a single interaction.

2. Long-Term Memory: Cross-Session Knowledge Retention 📚

This category is further broken down into several types:

  • Semantic Memory:
    • Human Analogy: Our long-term store of facts about the world or learned knowledge (e.g., how to navigate a subway map).
    • For Agents: This includes knowledge bases, proprietary information, or facts gathered about users and other entities.
  • Procedural Memory:
    • Human Analogy: Knowing how to do things without conscious thought, like typing or riding a bike (muscle memory).
    • For Agents: This can come from LLM training (weights), instructions in prompts (prompt engineering), markdown files, skill spec files, or logging steps taken to solve complex problems for future reference.
  • Episodic Memory:
    • Human Analogy: Remembering key life events, like graduating from university.
    • For Agents: This captures key events or interactions with its user environment. For example, a travel agent remembering a user’s negative review of a restaurant to avoid similar suggestions in the future.
  • Working Memory:
    • Human Analogy: Holding a small amount of information for immediate use, like intermediate results while solving a math problem. It’s the active workspace, not storage.
    • For Agents: This is essentially the LLM’s context window, a mix of current input, recent session history, and retrieved long-term memory. Some agents also use a scratchpad – a temporary file for noting intermediate work.

Hands-On Implementation with MongoDB: Building Your Memory System 🛠️

The workshop then transitions into practical implementation, guiding participants through:

  • Setting up memory collections in MongoDB: Creating dedicated spaces for different types of memory.
  • Creating indexes: Ensuring efficient querying of these collections.
  • Persisting memory: Understanding how to create, read, update, and delete memories.
    • Creating Memories: Driven by prompts, LLM reasoning traces, tool calls, user conversations, or environmental feedback. Data modeling in MongoDB is crucial here, leveraging its flexible JSON-like document structure.
    • Retrieving Memories: This can happen at various points in the workflow depending on application needs. MongoDB offers text search (exact match), vector search (semantic similarity using embeddings), and hybrid search (combining both).
    • Updating Memories: Options include overwriting existing data or implementing temporal versioning using timestamps.
    • Deleting Memories: Essential to prevent overwhelming the system and maintain performance. This can be time-based (TTL indexes) or history-based (e.g., deleting memories with low importance scores). MongoDB also offers Atlas Online Archive for cold storage.

Key Takeaways for Designing AI Memory Systems 🔑

  • Memory is crucial for AI agents: It transforms stateless systems into stateful, intelligent ones.
  • Not all data is equal: Differentiate between short-term and long-term memory needs.
  • Human memory is a great model: Short-term, semantic, procedural, and episodic memory offer valuable frameworks for AI.
  • MongoDB is a powerful ally: Its flexible document model, indexing capabilities, and diverse search options (text, vector, hybrid) make it ideal for building robust AI memory systems.
  • Data modeling is key: Design your MongoDB collections to efficiently store and retrieve the right kind of memory for your application.
  • Application context matters: The choice of what memory to create, when to create it, and how to retrieve it is entirely dependent on your specific application’s needs.

By understanding and implementing these memory design principles, you can empower your AI agents to learn, adapt, and deliver truly intelligent and valuable experiences. Get ready to build smarter AI! 🚀✨

Appendix