Learning Byte / Learn

Please note, this video is best viewed with 1080p quality selected. You can use the player bar to navigate to chapters within the video.

Chapters

0:00 Intro

1:22 Prerequisites

2:03 Demo

10:40 Learn More

Resources

Download MongoDB Relational Migrator

To learn more and leave feedback:

    Video Transcript (English)

    Hi, and welcome to MongoDB Learning Bytes, where you can pick up knowledge on MongoDB in 20 minutes or less. I'm Scott Amerman, a Senior Product Marketing Manager at MongoDB. 

    In this Learning Byte, we'll go over Relational Migrator, a tool which simplifies migrating relational workloads to MongoDB. By the end of this course, you should have a good understanding of what Relational Migrator is and how to get started using it to design a MongoDB schema and to migrate your data.

    Migrating from relational databases to MongoDB isn't always an easy task. With Relational Migrator, we aim to make three of the most common challenges a little bit easier. 

    Relational Migrator helps you design an effective MongoDB schema by providing recommendations based on an existing relational schema. These recommendations will help you simplify your data model and take full advantage of the document model.

    Second, Relational Migrator can create DataSync jobs to bring your data from a relational database to any MongoDB deployment while transforming it to the target schema you designed in the tool. And finally, data migration is just one part of the equation. 

    Your application code needs to change too. Relational Migrator generates code artifacts that can give you a head start with updating your applications to work with MongoDB.

    Before we get started with this tutorial, you're going to need a few things. Relational Migrator is a downloadable software that is available on mongodb.com and can be run locally on your device or server. Then, you'll need an existing relational database to migrate from.

    Today, Relational Migrator supports Oracle, SQL Server, PostgreSQL, MySQL, and Sybase ASE, also known as SAP ASE as your database sources. You will also need a MongoDB database like Atlas to migrate to with a user that has read and write access. For both databases, you'll also need connection details.

    Let's walk through a small migration from beginning to end using Relational Migrator.

    We'll start by connecting to a relational database, generate a recommended MongoDB schema, make some changes to that schema, and then migrate that data to MongoDB Atlas. We'll start by creating a new project and connecting to a live relational database. 

    We could also start with an SQL file or use a sample schema if we're looking to experiment with the design portion of the tool first. In this example, I will select PostgreSQL and manually add the URI.

    You could also use the fields below to have Relational Migrator construct it for you. I'll add my username and password and test my connection to make sure I've done everything right. Everything looks good, so let's go ahead and connect to our database. 

    Now, select the tables we want to bring in from our relational database.

    I'm going to open up the Northwind row here. This is a demo database that mimics a retail application including tables like customers, their orders, and the products that go in those orders. 

    We can select every table or exclude ones we don't want to migrate, which is handy for extremely large schemas. Next, we define our starting MongoDB schema.

    Relational Migrator has three options available. By default, it creates a one-to-one schema where every table starts as a top-level collection. You can also start with a blank canvas where you manually create the new schema. 

    Finally, Relational Migrator can also suggest a recommended schema based on the relationships in your existing data.

    When the recommended schema option is selected, you can see recommendations below for what data should be top-level collections and what data should be embedded. 

    For a real project, you'd probably start with our recommendations and make changes before migrating. But let's start with a blank schema so we can see how to build a MongoDB schema from scratch. Let's take a look at our relational model.

    We have an orders table, which is linked to several tables, like order details, the customers who bought those orders, and the employees who are fulfilling that order. We'll use the mappings panel on the right side to turn our orders table into a new top-level collection. 

    Now, when we click on our new collection, we can see Relational Migrator making recommendations for how tables should be mapped. Instead, let's go back to our relational model and look at our order details table.

    Orders has a one-to-many relationship to order details, so we can put this data into our orders collection. We can map this as an embedded array with the parent collection orders and even remove fields, like IDs, that we won't need in MongoDB. 

    Now, let's take a look at the products table, which includes data about product names and a lot of inventory related data points like stock and prices. Let's add it as a new collection using the same method as the orders collection.

    However, a real world use case may want product summary information, like product name and price, on every order we collect. To do this, we will add products as an embeddable document to our orders collection. 

    We won't need every field from the products table, so let's uncheck everything else and select the fields we do want. In the real world scenario, this is where we would continue mapping fields from the relational schema to our MongoDB schema, making simplifications and changes as we go.

    The goal is to redesign our schema so our application takes full advantage of MongoDB's document model instead of just bringing our relational schema over as is. 

    MongoDB has a ton of resources available to help you learn effective data modeling, and Relational Migrator schema recommendations and recommended mappings should help you out on that path. 

    For the tutorial's sake, we'll stop here and just assume that we have an effective MongoDB schema so we can move into creating a migration job. I already have a fresh Atlas database created with no data in it.

    So I will click on the Data Migration tab to create a new sync job. Relational Migrator will remember our relational database connection details so we can skip that step. But if we need to, we could choose a different database server. 

    This would be useful if we're designing a schema against a development database, but we actually wanted to migrate data from a staging or production environment.

    Now, we'll connect to our MongoDB database. If you're following along with the tutorial, I recommend spinning up a new MongoDB Atlas account if you haven't already, but you could also connect to any self-managed MongoDB database as well. 

    We can then choose if we want to run a snapshot sync or a continuous sync job. Continuous sync jobs are ideal for migrating without downtime.

    This mode migrates the data, then goes into a watching state. If you make any changes to your relational data, Relational Migrator will see those changes and push them to your MongoDB database. 

    For long-running continuous sync jobs, we recommend connecting to a Kafka Cluster. Relational Migrator makes it easy to connect to Kafka via Confluent Cloud and Confluent Platform, or you can bring your own Kafka Cluster.

    For today, we'll just do a simple snapshot job. This demo migration is a relatively small data set, just a few thousand rows, so it should only take a few seconds. And we're done. Let's go back over to Atlas to see our data.

    And there it is. We can see our orders collection with our order details and products embedded in it. 

    Hopping back over to Relational Migrator, let's take a look at two features that can save you time when you're ready to refactor your applications. On the Code Generation tab, you can generate application code # for several common languages and frameworks, including C#, Java, JavaScript, and JSON.

    Just click the language your application is using, the template you need, and voila, code that reflects your new MongoDB schema will be available in the windows and tabs on the right. 

    You can copy this code directly or download everything as a zip file. While this code isn't necessarily production ready, it should still reduce the time spent on development activities you're going to have to do anyways. You'll also find the Query Converter under the Code Generation tab.

    This tool uses generative AI to help you quickly convert existing SQL queries, triggers, and stored procedures to work with MongoDB using your choice of programming language, # including JavaScript, C#, and Java. We'll start by importing the stored procedure from the relational database into our Relational Migrator project. 

    This particular stored procedure joins the results from two tables, performs some arithmetic on some of the columns, and filters the results based on an input parameter. 

    For developers who have experience working with the MongoDB Aggregation Framework, you'll notice that this is the equivalent to joining data from two collections using the $lookup stage.

    However, when migrating to a relational database to MongoDB, it often makes sense to consolidate data from multiple tables into a single collection because data that is accessed together should be stored together in MongoDB. 

    In this example, we've already done that in the schema combining data from orders, order details, and products table into a single orders collection. 

    This means that when considering the changes to the schema, we do not actually need a $lookup stage at all, as the data from each of the required tables has already been merged into a single collection. 

    Relational Migrator's Query Converter works alongside the schema mapping functionality and automatically adjusts the generated query to work against your chosen schema.

    With JavaScript chosen as our target language, the converted query avoids the need for costly join and includes MongoDB equivalents of our original SQL arithmetic functions. 

    The query is now ready to test and include in our modernized app. Now that you've seen the main features of Relational Migrator in action, try using it on your own data to connect to an existing relational database, design an effective MongoDB schema using your relational schema as a starting point for recommendations, and migrate your data from your relational database to MongoDB.

    Relational Migrator is a powerful tool, but to make sure you have a successful migration project, we recommend exploring a variety of resources to help prepare.

    MongoDB University has a variety of courses geared towards teaching you about MongoDB, including the MongoDB to SQL Professionals Learning Path and the MongoDB Data Modeling Path. 

    MongoDB can also help you with your migration initiatives. Visit mongodb.com to learn more about professional services and our partners that can help you with everything from MongoDB migrations to application development.

    If you have any feedback on this Learning Byte or Relational Migrator, you can submit them on the MongoDB feedback engine at feedback.mongodb.com.