We’ve already discussed the DBMS (software that runs the database), but just like all other software, the DBMS requires hardware to operate. With modern databases, there are a lot of options for not only the hardware specifications, but where that hardware exists.
Modern databases are required to support critical applications around the world for millions of users. In order to serve as the backbone for these critical applications, there is an expectation that there cannot be downtime or loss of data. For instance, imagine if a banking app missed a user’s transaction. Missing a deposit or a credit comes with significant cost, literally. Having the app down for any amount of time could also prevent the user from being able to access their account. Neither of these scenarios are acceptable to the bank or the user.
While not all applications are critically important, having the ability to keep the database up and running is a minimum requirement. In order to satisfy these requirements and meet the growing demand for data storage, MongoDB uses a distributed architecture to expand capability and ensure performance no matter what. Let’s explore some of the key strategies MongoDB uses to deliver those results!
The concept of scaling a database is critical to ensuring you can meet the needs of users. Scaling refers to adjusting the hardware supporting the database in order to accommodate a changing number of users or operations. “Scaling up” means adding hardware capacity while “scaling down” means reducing the hardware available.
Scaling is critical to efficiently hosting a database because if you were to allocate too few resources, you may miss transactions, have poor performance and user experience, and even risk crashing the database altogether. On the other hand, deploying too much hardware for an instance will result in additional costs and no performance gain, which is also undesirable.
Properly allocating hardware for databases has gotten more complicated over time. General purpose databases were originally designed to run and serve data from a single server. This is related to the fact that they were designed in the 1960s, but this has persisted over time because there are also some performance benefits to locating data on a single machine. With the growth of the Internet and the massive numbers of smart devices, more data is being generated and stored, making it harder to determine how much capacity is required.
To better understand scaling, let’s pretend we run a web application that has a fairly predictable number of users per day, say 5,000 users. If we were to run a promotion or sale we may expect additional users to use the application because they want to take advantage of the limited-time offer. During these spikes, we may have twice as many users per day, which would definitely require us to scale up our hardware, otherwise we risk losing out on customers. Let’s look at methods for scaling and discuss how we’d use them with our example.
There are two methods of scaling hardware: Vertical and Horizontal.
In our application example, we need to acquire hardware that can support the maximum number of users we expect (10,000). That means that when we are not having a promotion, our more expensive hardware will be at half capacity. On premises vertical scaling is inefficient for this example because we can’t scale down for times when we have less volume.
With the creation of the cloud in the mid 2000s, the need for hosting and managing your own servers has changed. Cloud provisioning is a term that describes reserving resources in the cloud. This can include CPU, storage, memory, and networking resources. Because these resources are virtually provisioned, it’s possible to scale up or down easily. This makes horizontal scaling a very effective way to meet demand without over-spending. Note: The cloud also makes vertical scaling easy as you have access to the latest hardware with less upfront investment required.
Another feature that is worth noting is replication, which is the process of storing multiple copies of data and keeping them synchronized across multiple nodes to provide redundancy. Like with sharding, the database is split over multiple machines, but for a different purpose. Replication increases resiliency and availability. Because the same data is stored on multiple nodes, the database will continue to run as expected even if one of the nodes experiences a failure and goes offline.
In MongoDB, the term “replica set” refers to a group of nodes that all have the same data. Replica sets should consist of an odd number of nodes with a minimum of three nodes. One of those nodes is designated as the primary node and the rest secondary nodes. Since replica sets maintain multiple nodes with redundant data, you may be wondering how data stays consistent across nodes. After all, data being written will likely be written to one node first.
When writing data on a distributed system, it can be a challenge to keep data consistent across the nodes. MongoDB manages data consistency through Read and Write Concerns, which are rules that specify how the database should respond to CRUD operations. Ultimately, the database administrator chooses a tradeoff between different aspects of performance when configuring these rules.
In the video below, you’ll learn about the CAP Theorem which describes the performance tradeoffs that are controlled by your Read and Write Concerns in MongoDB.
As you can see, there are a lot of factors to consider when planning and configuring a database. Ultimately, the database administrator chooses what aspects of the CAP Theorem to prioritize given their expected workload and application importance. With the cloud and modern hardware, we have more choice than ever when deciding how we want to address performance tradeoffs. No matter what that balance is for your application, MongoDB can accommodate your needs with its distributed data architecture.
In this section we covered some of the key concepts that allow distributed databases to meet the demands of modern applications. Here are some key takeaways:
- Scaling: You can adjust the performance of the database using more performant hardware (vertical) or by adding additional hardware (horizontal).
- Replication: By creating redundant nodes, we can ensure availability and protect from data loss in the event of node failure.
- The CAP Theorem: There is always a tradeoff between Consistency, Availability, and Partition Tolerance.