Query Optimization / Optimize Query Performance
If you're trying to fix something, it's important to have the right tools for the job. But having the right tools means nothing if you don't know how to use them. In this video, we'll cover some strategies and tools to fix underperforming queries.
Let's begin with a quick recap of the query tuning life cycle. The three phases are identify, analyze, and optimize.
The identify phase is where we start pinpointing problem areas. In the analyze phase, we examine the problematic queries. And the optimize phase is where we make changes.
In the identify phase, to look for slow queries, we use tools such as the log files, the query profiler in Atlas, and the database profiler in the MongoDB shell. The simplest way to find slow queries is to use command line tools. For instance, you can use the grep command to find log entries that contain slow query in them. This is because MongoDB logs queries that exceed a certain execution time. This is known as the slow ms threshold.
By default, this threshold is set to one hundred milliseconds, but we can change that threshold based on our business requirements.
Once a query exceeds the slow MS threshold, MongoDB logs important details about it, such as the specific query being executed, the execution time, any indexes used, the number of documents examined versus returned, and other information.
This info is crucial for figuring out why a query or a query pattern might be underperforming and the best strategy for fixing it. We can also profile our queries to find ones that are performing poorly.
MongoDB provides various tools such as Query Profiler in Atlas and Database Profiler in MongoDB Shell. Like the log file, these tools capture valuable details about query performance that can identify problematic queries. The query profiler, also available in Atlas m ten or higher clusters, captures metrics such as execution time, number of documents examined, and the number of documents examined to return ratio for each operation.
It also offers other granular insights and supports historical analysis, helping to identify bottlenecks and track the impact of optimizations over time.
Atlas displays this data on the query profiler tab in your query insights section on your Atlas dashboard.
If you're not using Atlas, you can use the database profiler. It's accessible from the MongoDB shell and collects information about database commands executed against a running MongoDB instance. The database profiler records similar metrics as the log files, making it a good alternative if the logs are not easily accessible from your machine.
We can configure the profilers logging level, and its data is stored in the system dot profile collection, making it easy to query and analyze the data directly from the shell.
The next phase is the analyze phase.
To analyze poorly performing queries, we can use explain plans.
Explain plans are a fantastic resource that give insight into how MongoDB executes a query, the execution path, and any indexes used. By analyzing the output of the explain function, we can understand the efficiency of our queries. The output includes which indexes were used, the number of documents that were scanned, and whether the query was covered by an index or not. Once we've analyzed our poorly performing queries, we can proceed to the optimize phase.
We can accomplish this in a variety of ways, adding or modifying indexes on our collections, forcing the use of specific indexes for a query shape, altering our queries, and altering our data model. To achieve this, we can leverage a number of MongoDB tools, such as the performance advisor in Atlas, as well as the create index command and query settings available in the MongoDB shell. Adding or modifying indexes in Atlas M10 or higher clusters is made easier by the performance advisor tool. It identifies slow running queries and provides recommendations for enhancing query efficiency, such as adding or dropping indexes.
If we're not using Atlas, we can still add indexes to our collections from the MongoDB shell. We can create indexes for a collection by using the create index command. We pass in the field names that we'll include in the index followed by either a one for ascending or minus one for descending.
These values determine the order that fields are stored in the index.
Either sorting order can be used for both ascending and descending sorts. But if we're using compound indexes to optimize a query, it's best to sort the fields in the manner that you plan to sort your queries in. If indexes are not being used properly to optimize query performance, we can force the use of specific indexes on the server without having to alter our code base. We can do this by utilizing query settings to optimize the query execution.
Query settings allow us to include index hints for specific query shapes.
These settings persist across cluster shutdowns and are used by the query planner during plan selection. If these attempts at optimization still result in poor performance, we may need to look at altering our queries to better align with our schema or alter the data model itself.
Though this approach is intensive, it aligns data modeling with query patterns and improves efficiency.
Well, we just talked about a lot of different strategies and tools, and you're probably wondering how to apply them all. Don't worry. We'll dive deeper into these tools later. Let's recap what you learned. First, we explored the query tuning life cycle, which involves three phases, identify, analyze, and optimize.
In the identify phase, we use strategies like scanning log files for slow queries and profiling queries with MongoDB tools such as the query profiler in Atlas and the database profiler in the MongoDB shell. During the analyze phase, we learned that we can utilize explained plans to gain insights into query execution paths, including index usage and document scan counts. This analysis helps us understand query efficiency.
In the optimize phase, we enhance query performance by adding or modifying indexes, forcing specific index usage, altering queries, or modifying the data model. The performance advisor in Atlas can automate some of these optimizations by suggesting index improvements for slow queries. If you're not using Atlas, similar tasks can be performed using the MongoDB shell. Query settings allow inclusion of index hints for specified query shapes and persist across cluster restarts.
By applying these tools and strategies, we can keep our databases responsive and efficient even as application demands grow.
