Video Transcript (English)

How does an agent remember what you said just moments ago? If you ask about the weather in Paris, and then follow up with "What about the temperature?"—how does the agent know you're still talking about Paris?

That's short-term memory in action. It's what keeps a conversation coherent and allows an agent to track context without you having to repeat yourself.

In this video, we'll first define short-term memory and see why persistence matters.

Next, we’ll focus on working memory that handles active agent cognition during a task. Then, we’ll explore how semantic caching acts as a complementary optimization layer, speeding up the retrieval of frequently used data to support that memory.

After that, we'll examine real-world use cases where short-term memory shines. Finally, I'll explain why we're using MongoDB to implement it.

So what exactly is short-term memory? It’s where the system holds onto the 'who, what, and where' of your current session so it doesn't have to start every sentence from scratch.

When you're having a conversation with an agent, short-term memory holds the context of that conversation. It remembers what you asked, what it answered, and details you've shared along the way. If you mention your name at the start of a chat, short-term memory is what enables the agent to use it naturally throughout the session.

For example, you ask: "What's the weather in Paris?"

The agent responds with the forecast. Then you follow up with "What about the temperature?"

Without short-term memory, the agent wouldn't know what "the temperature" refers to. But with it, the agent understands that you're still asking about Paris.

But, for short-term memory to be useful, it needs to survive restarts within a session. If our application crashes or the user refreshes the page, the conversation shouldn't disappear.

To achieve this persistence, we need a file system or a database to store the information in our short-term memory.

Now that we have a basic idea of what short-term memory does, we can look at its primary function, working memory.

Think of working memory as the agent's "scratchpad," where it actively handles information during a task. It's a dedicated area in the LLM's context window, or a temporary file, that only lasts for the current session. It keeps track of chat history and lets the agent update its memory state in real time as the conversation moves forward.

Think about a coding assistant. When we're writing a function, the assistant needs to remember the variables we've declared, the function parameters, and the context of what you're building. The working memory lets the assistant track multiple pieces of information and use them together.

Working memory enables coherent multi-step reasoning by allowing an agent to maintain a consistent "thread of thought" across a sequence of operations.

It's the foundation for tasks like data analysis, report generation, or any workflow where the agent needs to build on previous steps.

Complementing working memory is semantic caching, which acts as an optimization layer for reusing recent responses efficiently.

Traditional caching matches exact queries. If someone asks "What's the capital of France?" and we've cached that answer, we can return it.

But if someone says "What city serves as France's capital?" Well, that's an entirely different string, so a traditional cache misses it completely.

On the other hand, semantic caching stores and retrieves responses based on the meaning of the query, not the exact words.

It understands that different phrasings can mean the same thing. So both questions about France's capital could return the same cached response.

Why does this matter?

Because it improves response times and reduces computational costs. Every time you can serve a cached response instead of making a new LLM call, you save time and money.

For applications where users frequently ask similar questions in different ways, semantic caching can make a difference.

Now that we’ve defined working memory and its optimization through semantic caching, let's see how these systems work together in the real world.

The first use case is Conversational context. This relies on Working Memory to maintain the flow of a conversation across multiple turns. When a user asks follow-up questions, the agent uses its 'scratchpad' to understand what came before so it can provide a coherent answer.

We also see working memory in action during multi-step tasks. For complex workflows like data analysis or report generation,the agent needs to remember intermediate results as it works through the problem. This lets it build on its own progress step-by-step, rather than starting from zero every time it moves to a new line of code or a new paragraph.

Working memory is also critical for Error recovery. If a failure occurs during a complex task, the agent uses its scratchpad to 'remember' where it was, allowing the agent to retry or adjust without losing all progress.

There's also contextual grounding. This involves using the working memory to keep track of entities, references, and topics mentioned in recent exchanges. If a user mentions "the document" or "that API," the agent knows what they're referring to because it has the recent context stored.

Finally, for high-traffic applications, we use Semantic Caching to handle Efficiency. By recognizing when a new question has the same meaning as a previous one, the agent can pull a ready-made answer from the cache. This reduces response times and cuts down on computational costs.

All of these use cases share one thing in common: they depend on the agent having access to recent information that wouldn't fit in a single context window or that needs to survive beyond one API call.

Now that we see how working memory powers active agent cognition, the question becomes: how do we store that memory reliably?

While a simple file system might work for a prototype, an intelligent agent requires a more robust foundation. Let’s look at why we would choose MongoDB for managing short-term memory needs.

The first advantage is the MongoDB document model. Conversation state naturally maps to JSON-like documents, meaning we don't need rigid schemas or complex table structures.

Because short-term memory includes everything from metadata and timestamps to tool outputs, we need a flexible schema like MongoDB's to handle it all in one place.

A binary representation for the current state of an agent is stored as a document that can evolve as our session grows.

For example, if we suddenly need to store a user’s preferred 'tone' or 'language' mid-session, we can add those data points in the current state to the document instantly without the headache of migrating an entire database schema.

Secondly, MongoDB provides a unified platform for all our memory needs.

Instead of jumping between entirely different database providers, we can manage our active session state through a checkpointer and our long-term vector embeddings all within MongoDB.

This means we don't have to worry about syncing data across different systems. Whether we perform a 'semantic cache' lookup or loading user preferences, we're hitting the same reliable infrastructure, which reduces complexity, lowers latency, and makes the agent feel much more responsive.

Great Job!

In this lesson, we explored short-term memory in AI applications. We defined it as the agent's working space that persists within a session, even through restarts.

We compared two implementation approaches: file systems, which are simple but limited, and databases, which provide the durability and concurrency control needed for production.

We also explored working memory for active processing and multi-step reasoning, alongside semantic caching, which serves as a complementary optimization for efficiently reusing responses based on meaning.

We also covered key use cases like conversational context, multi-step tasks, session personalization, error recovery, and contextual grounding.

And finally, we saw why MongoDB is a strong choice for implementing short-term memory, with its flexible document model, unified platform, and efficient indexing.