Query Optimization / Identify and Resolve Slow Queries
Now that you know what query tuning is, how MongoDB's architecture ties in, and some strategies for optimizing queries, let's apply the query tuning life cycle in action.
In this video, we'll identify a slow query and analyze it. We'll use the MongoDB Atlas query profiler and log data to identify and analyze slow queries in a cluster. You recall that MongoDB Atlas offers a query profiler that helps identify slow queries. It provides insights into query performance so that we can focus on the queries that need attention. The query profiler is available in m ten or higher clusters.
To access the query profiler, we first need to log in to our MongoDB Atlas account.
From here, we can select the cluster we want to analyze.
Then we navigate to the query insights tab.
The query insights tab is available for m ten or higher clusters and provides detailed visibility into the performance of your database queries.
From this tab, we select the option for query profiler.
The profiler gives a detailed view of the queries being executed, including metrics such as execution time, the number of documents examined and returned, the ratio of documents examined to returned, and more. This information is crucial for identifying which queries are slowing down your application. First, we want to look for queries with high execution times, queries that are taking longer than what our business needs require.
Execution time is a top indicator of a query's performance because it directly impacts the user experience and system efficiency.
We also want to look for queries that we notice are executed frequently.
So how do we identify queries that fall into these categories?
To help find our queries with high execution times, we want to make sure that operation execution time is selected.
Next, we want to narrow the scope of our graph to specific time periods to focus on when performance issues occur. For example, if problems have been reported in the last hour, we choose one hour from the list of options. We can expand our window up to five days from this list. And if we wanted to troubleshoot a previous issue, we can click here to filter by a specific time range.
If our cluster has multiple nodes, we can also filter by host to see where the slow queries are happening.
We can see a few slow operations on our graph. To further isolate our problem queries, we can click and drag on the graph to zoom in on a particular time window to get a more detailed analysis.
Ah, that's better. If we highlight a data point, we can see specifics about the database and collection involved in the queries.
This gives us some quick insights into potential causes for the slow operation.
For example, we can already see that this query used a collection scan and how long it took and that it used a sort stage. From here, we can click on view more details to examine the slow operation more closely.
This takes us to a page for our slow queries with a specific query already selected.
Here, we are immediately given valuable information about the query, including operation execution time, keys and docs examined, docs returned, and examined to return ratio.
Let's start our analysis by going through each of these metrics and talk about why you may be interested in them. The operation execution time refers to the total time taken for a query to execute from start to finish. This includes the time spent processing the query, accessing the necessary data, and returning the results.
Our goal for this exercise is to have the execution time less than one hundred milliseconds, but the target time should be whatever your business requirements are. Key is examined refers to the number of index keys that MongoDB evaluates during the execution of a query. This metric is relevant when a query uses an required documents.
This query shows zero keys examined, which means no index was used. We can see the effects of no index being used on the next two metrics.
The first of those is docs examined, which refers to the number of documents that MongoDB scans to fulfill a query. We can see our query examined a lot of documents. But by itself, this is not a direct indicator of poor query performance.
We can evaluate the performance by comparing this number to the docs returned value. The next metric is docs returned, which refers to the number of documents that a query returns as its result.
This metric helps you understand the output size of a query. The output for our query is fairly small. This means we're examining a lot of docs for every return doc, which is not ideal.
Finally, we have the examine to return documents ratio metric. This ratio can indicate inefficiencies in query execution.
A high ratio suggests that MongoDB is examining many documents but returning few, which can be a sign that an index is needed. The ideal ratio is one to one. Our examined to returned ratio is significantly high. This means we either have a poorly designed index or no index at all.
To dig deeper into this, let's look at the log information for the query. On our query details page, we see the parsed log document. This document eliminates the need to download log files for the purposes of examining the slow query. When we look closely at the log document, we see the query structure.
We're matching on address dot market and amenities fields.
Then we're doing range scans on the price, minimum number of nights, and bedrooms.
Finally, once we have all that data, we're sorting by price from highest to lowest. If we move further down the log information, we can see that the query plan used a collection scan to retrieve the data. Based on this analysis, we can assume that the first step to optimizing this query will be to add an appropriate index to the collection so we can leverage the index instead of scanning the entire collection.
Great work. In this video, we walk through the process of identifying and analyzing slow queries using tools in MongoDB Atlas. Using the query profiler and log data, we pinpointed queries that needed to be tuned and gained insights into how to improve them.
