Video Transcript (English)

You added vector search to your app and it's blazing fast on your laptop. Queries return in milliseconds, everything feels instant. Staging looks fine too -- but a week into production, query latency starts climbing.

Then the tickets start coming in. Users are reporting slow responses, and you're not sure where to look.

In this video, we'll explore how MongoDB Vector Search works under the hood: how vectors are stored, how the default HNSW index type finds results, and why RAM is one of the biggest factors in query latency.

Before we talk about what's happening in production, we need to understand the data vector search is working with.

A vector is a list of floating-point numbers produced by an embedding model. When you pass a piece of text, an image, or any other data to an embedding model, it outputs a vector that encodes the semantic meaning of that data as a point in a high-dimensional space.

Two pieces of data that are semantically similar will produce vector embeddings that are close together in that space.

Within a vector, each number in the list is called a dimension. A vector from Voyage AI's `voyage-4` model has 1,024 dimensions by default. More dimensions give the model more expressive power to capture meaning, but they also mean more storage and more computation per vector.

Now that we know what vectors are, the next question is: how do you search through millions of them fast enough for the search to feel instant?

The answer is an algorithm called HNSW, which stands for Hierarchical Navigable Small World. Rather than comparing your query vector against every single stored vector, HNSW builds a multi-layered graph where each node represents a vector and connects to its nearest neighbors.

When a query comes in, the search engine enters the graph at the top layer and traverses downward, hopping between nodes that are progressively closer to the query vector — finding approximate nearest neighbors in far fewer comparisons than a brute-force scan.

This is why vector search can return results in milliseconds even over large datasets. But there's a catch — one that becomes very visible when your queries are exposed to production workloads where the index size tends to be larger.

MongoDB Vector Search depends on a two-process architecture to work. It runs through `mongot`, a dedicated search engine process that owns and manages all vector and full-text search indexes — including the HNSW graph. But `mongot` doesn't operate in isolation. It relies on `mongod`, the core database process, to receive queries, coordinate results, and return data to your application.

When a `$vectorSearch` query comes in, here's what happens.

First, your application generates a query embedding and sends the aggregation pipeline to `mongod`, which proxies it to its associated `mongot` process.

`mongot` searches its HNSW index to find the closest stored vectors according to the similarity measurement and returns the matching document IDs and search metadata back to `mongod`.

Finally, `mongod` performs a full document lookup on those IDs and returns the results to your application.

Notice that your application never talks to `mongot` directly — all communication flows through `mongod`. But `mongot` runs on the same node as `mongod` by default, which means both processes are drawing from the same pool of provisioned RAM. That shared memory is where the contention lives — and understanding how it's divided is key to understanding vector search performance.

Here's how that RAM is used. On M40 and larger clusters, mongod's WiredTiger storage engine claims roughly 50% of available RAM for its database cache -- on M30 and smaller, that drops to 25%. That chunk is effectively reserved for `mongod` to hold frequently accessed data. The remaining 50% is shared between additional operations. These include `mongod` operations like sorting, connection handling, and aggregation, `mongot`'s JVM heap and `mongot`'s index data.

Regular mongod operations are critical but they do not involve Search indexes so let's focus on mongot behaviour.

Search Index data managed by 'mongot' lives in the OS page cache. Under the hood, `mongot` stores your vector and full-text search indexes as Lucene index files on disk. Lucene is the search library `mongot` is built on, and it accesses those files using memory-mapped I/O, meaning the OS treats them as if they were directly addressable in memory.

As mongot reads a vector search index during HNSW traversal, any index pages that are not already resident in memory are loaded from disk into the OS page cache.

If the pages are already cached, the OS serves them directly from RAM without additional disk I/O.

The page cache isn't a fixed allocation, though. It's dynamic: the OS fills whatever free memory is available, and under memory pressure it will evict pages to make room. `mongot`'s Lucene index data is a candidate for eviction just like anything else. When those pages get evicted, the next HNSW traversal has to fetch them from disk.

So `mongot`'s total memory footprint is its JVM heap plus however much of the page cache your search indexes are occupying. Both draw from the same remaining pool after WiredTiger takes its share.

Now that you understand the contention, let's try to understand just how big a vector search index can be.

Every vector in your index takes up space in memory, and that space is driven by two things: the number of documents in your collection that have vectors and the number of dimensions your embedding model produces per vector.

To get a rough sense of scale, let's walk through a back-of-the-envelope estimate using float32 vectors.

Each dimension costs about 4 bytes, so for a 1,024-dimension model, a single vector takes up roughly 4 kilobytes. This means that a collection of 100,000 documents is around 400 megabytes, a million documents around 4 gigabytes, and 10 million around 40 gigabytes.

These are intentionally simplified numbers — your actual footprint will vary based on your model's dimensionality, your data, and HNSW graph overhead on top — but they give you a sense of how fast the math compounds as collections grow.

We can express this as:

index_size_bytes is approximately equal tonum_vectors times dimensions timesbytes_per_value

This is a rough estimate, not an exact figure, but it's close enough to size your infrastructure and spot potential problems before they happen.

So why does all of this point back to RAM? It comes down to how HNSW traversal works at a low level.

When `mongot` traverses the HNSW graph during a query, it hops between nodes in an essentially random access pattern.

Each hop lands at a different location in memory, and at each stop it needs to read a vector value to compute similarity. If that value is in RAM, the read takes nanoseconds.

If it has to be fetched from disk, that same read takes microseconds. That's 1,000 times slower. And because a single query may traverse dozens to hundreds of graph nodes, one slow disk read per hop adds up fast.

This is the RAM rule for vector search: the entire HNSW graph and your vector data need to fit in `mongot`'s available memory for consistently low-latency queries. The moment your index exceeds available RAM, `mongot` starts reading from disk, and query latency becomes unpredictable.

This is the thread that connects every technique we'll cover in this Skill badge. Quantization, partial indexing with views, dedicated Search Nodes — all of these are ultimately about one thing: keeping your index in RAM.

Awesome work! Before we move on to the techniques for optimizing vector search performance, let's recap what we've learned.

In this video, we covered what vectors are and how embedding models produce them. We also looked at how the HNSW algorithm enables fast approximate search through a multi-layered graph.

From there, we explored the two-process architecture of `mongod` and `mongot`, how node memory is divided between them, and why keeping your Lucene index data in RAM is the defining factor in query latency.