Vector Search Performance / Optimizing Vector Search with Views

10:03
Your collection has a million documents, but only a fraction of them are actually relevant to your vector search workload. What if you could build an index over just those documents without touching your source data? In this video, we'll introduce MongoDB views and show how they let you do exactly that. Build vector search indexes over a filtered subset of your collection. We'll also look at when a standard view is the right tool and when you need a materialized view instead. Before we look at how views fit into a vector search optimization strategy, let's make sure we're on the same page about what a view actually is. A view is a named read only aggregation pipeline stored in MongoDB. When you create a view, you give it a name and define a pipeline, a series of stages that transform or filter documents from an underlying collection. MongoDB evaluates that pipeline on demand when you query the view or when mongot builds an index over it. The source collection is never modified. For vector search, this matters because you can create a vector index directly on a view. When you do, MongoDB only indexes the documents the view produces, not the full collection. Whatever your pipeline filters out never enters the HNSW graph at all. One version note: On MongoDB 8.0 (on Atlas), indexes on views must be created via the Atlas UI or administration API and vector search queries must target the source collection by name. Starting with MongoDB 8.1+, mongosh and driver methods work as well and you can query the view directly. Now that we know what a view is, there's an important distinction to understand before we start building. MongoDB has one native view type, but there's also a common pattern for simulating materialized views. And the choice between them affects how your vector index gets built. The first is a standard view. A standard view is just a pipeline definition. Nothing is written to disk. Every time mongot indexes or replicates changes, it evaluates the pipeline against the source collection and works with the output. This means the view is always fresh and requires no extra storage. For vector search, standard views work well when your pipeline is lightweight. A $match to filter documents, a $addFields to reshape a field, or a typecast to make a field compatible with the index. Keep it simple and deterministic, and you won't run into problems. Next, we have materialized views. Unlike standard views, materialized views aren't a native MongoDB feature. They're a design pattern you implement yourself. You run an aggregation pipeline using $merge or $out to write the results to a real collection and then build your vector index on that collection. Because the output is a regular collection, MongoDB indexes it directly without evaluating any pipeline at index time. This decouples the cost of your transformation from the cost of building and maintaining the index. The practical guidance here is straightforward. Use a standard view when your pipeline is simple, a small number of stages, no joins, no complex expressions. Use a materialized view when your pipeline is doing significant work, like multistage lookups, aggregations that touch a lot of data, or when it uses pipeline stages that vector search on standard views doesn't support. Standard views only support $addFields, $set, and $match with an expression operator. Anything else requires a materialized view. In those cases, paying the pipeline cost once during a scheduled refresh is far better than paying it on every index replication event. For the rest of this lesson, we'll be focusing primarily on standard views. They cover a good portion of real world use cases and are the right starting point for most vector search optimization work. We'll call out when a materialized view is the better fit, but the patterns we build will use standard views. Let's look at the first practical use case, filtering out documents that don't have embeddings yet. In most production systems, your embedding pipeline doesn't run instantaneously across the entire collection. New documents arrive, your pipeline processes them in batches, and for some time, a portion of documents simply don't have an embedding field yet. If you build your vector index directly on the collection, mongot has to handle those embedding list documents and that wastes RAM on entries that contribute nothing to search results. A view solves this cleanly. We create a view with a match stage that only passes through documents where the embedding field is present. Let's see what this looks like. First, let's quickly connect to our Atlas cluster using the MongoDB Shell and switch to the database that we want to configure the view on. After that, we call the createView method on the database that holds the collection with our data with three arguments, the name of the view, the name of the source collection, and the pipeline. The pipeline here is a single $match stage. The expression operator ($expr) lets us use aggregation expressions inside the match, and $type returns the BSON type of the field. If the field is absent, $type returns the string missing. So this filter passes through only documents where the embedding field actually exists. Once this view is in place, you build your vector index on documents with embeddings instead of on documents directly. mongot only sees and indexes what the view produces. As your embedding pipeline catches up and populates new documents, the view automatically includes them on the next index update. No index rebuild required. The result is a smaller, more focused index that only holds vectors that are actually searchable. The same principle of filtering at index time, not at query time, applies to another common production scenario, multi tenant applications. MongoDB's recommended baseline for multi tenancy is a single collection with a tenant ID field on every document. It's operationally clean and avoids the complexity of managing one collection per tenant. But it creates a challenge for vector search. If you build one index over the entire collection, every query searches across all tenants' documents. That increases the candidate pool, raises the RAM footprint, and can reduce recall because the HNSW graph has to work harder to service the right neighbors. Views give you a way to solve this without abandoning the single collection model. The pattern splits tenants into two groups based on their data volume. For large tenants, you create a dedicated view per tenant with a $match on tenant_id, then build a separate vector index on each view. Each large tenant gets its own isolated HNSW graph, a smaller focused index where all candidates are already scoped to that tenant. For small tenants, you create a single shared view that includes all of them and filter by tenant_id at query time inside vector search. This avoids the overhead of maintaining a separate index for every low volume tenant. Here's what the large tenant side looks like in practice. The view filters the collection down to tenant A's documents only. The index is then created on that view using createSearchIndex on the tenant A docs view. From here, any vector search query run against this index is automatically scoped to tenant A. The HNSW graph only contains their vectors. One thing worth noting, as of MongoDB 8.0+, $vectorSearch cannot appear inside of views pipeline definition. You run vector search queries against a view and you build vector search indexes on views, but the view itself is just a filter and transformation layer. The search happens outside of it. The end result of this pattern is a set of smaller tenant scoped indexes that are cheaper to keep in RAM and faster to traverse, directly reducing the page fault risk we identified earlier in the course. Before we wrap up, there's one guardrail worth understanding. Not every aggregation pipeline is compatible with vector search index builds, And getting this wrong can leave your index in a broken state. When mongot builds or replicates a vector index on a view, it evaluates the view's pipeline against each document. If the pipeline includes an unsupported operator like random ($rand) or user roles ($$USER_ROLES), index replication will fail. Similarly, if the pipeline contains an expression that errors on certain documents due to unexpected data shapes or missing fields, mongot can get stuck, leaving the index in an incomplete or stale state until you fix the underlying issue. The rule to follow is keep View Pipeline's backing vector indexes simple and deterministic. A $match with expression, a $addFields to promote or cast a field, maybe a $set. These are the kinds of stages that work reliably. If your transformation needs are more complex than that, materialize the result into a real collection first and build the index there. This isn't a limitation to work around. It's a design signal. The view pipeline sits on the hot path of every index replication event. Keeping it lightweight means your index stays current and your search stays fast. Nice work. In this lesson, you learned how MongoDB views let you control exactly what gets indexed without modifying your source data. You saw how to use standard views to filter out documents missing embeddings and to scope indexes per tenant. Just remember, keep your view pipeline simple and deterministic, and your index will stay healthy.