Migrate a self-managed MongoDB replica set to ApsaraDB for MongoDB
Combine full and incremental data migration to move your self-managed MongoDB replica set data to ApsaraDB for MongoDB with zero downtime.
You can also use MongoDB tools to migrate a self-managed database to a replica set instance. Data migration and synchronization covers additional scenarios.
Prerequisites
The source and destination MongoDB versions must be compatible. Check the Migration solutions overview.ta#concept_26618_zh" baseUrl="t6682_v1_36_5.dita" data-node="17092">.
The destination ApsaraDB for MongoDB instance should have at least 10% more storage than the source database.raDB for MongoDB instance is 10% larger than that used by the self-managed MongoDB database.
Usage notes
Full data migration consumes resources on both the source and destination, which increases server load and may cause service disruptions. Migrate during off-peak hours if your database handles high traffic or runs on limited hardware.nal database. Do not migrate it unless necessary.
When DTS performs full data migration, it consumes resources of the source and destination databases, which may cause the database server load to rise. If the database workload is heavy or the server specifications are low, the database pressure may increase and the database service may become unavailable. Evaluate the impact carefully before migration, and perform data migration during off-peak hours.
e database server load to rise. If the database workload is heavy or the server specifications are low, the database pressure may increase and the database service may become unavailable. Evaluate the impact carefully before migration, and perform data migration during off-peak hours.
Check Versions and storage engines for supported MongoDB configurations. Verify compatibility before migrating across versions or storage engines.="t6682_v1_36_5.dita" data-node="6653">. If you need to perform cross-version or cross-engine migration, confirm the compatibility in advance.
The admin and local databases cannot be used as source or destination databases.
cannot be used as source or destination databases.
DTS writes data to the destination concurrently, which may cause the destination to use 5%–10% more storage than the source.e space occupied by the destination is 5% to 10% larger than that of the source.
Ensure no documents in the destination share a primary key (
_idby default) with documents in the source. Duplicate keys may cause data to be overwritten. Delete duplicates in the destination before migration if doing so does not affect your business.Otherwise, data loss may occur. If the destination has the same primary key as the source, clear the related data in the destination (delete documents with the same _id as the source) without affecting your business.
Billing
Migration type | Link configuration fee | Internet traffic fee |
Full data migration | Free of charge. | Charged only when data is migrated out of Alibaba Cloud over the Internet. For more information, see Billing overview. |
Incremental data migration | Charged. For more information, see Billing overview. |
Migration types
Full data migration: Migrates all existing data from the selected objects in the source MongoDB database to the destination MongoDB database.
NoteThis includes databases, collections, and indexes.
"note_wtw_n1y_f19">
Supports migration of database, collection, and index.
Incremental data migration: After the full data migration is complete, DTS synchronizes data changes from the source MongoDB database to the destination.
NoteSynchronizes
CREATEandDELETEoperations for databases, collections, and indexes.Synchronizes
INSERT,UPDATE, andDELETEoperations for documents.
s for databases, collections, and indexes. operations on database, collection, and index.
Synchronizes
INSERT,UPDATE, andDELETEoperations for documents.nsert, delete, and update operations on documents.
Database account permissions
Migration data source | Full data migration | Incremental data migration |
Self-managed MongoDB database | The read permission on the database to be migrated | The read permission on the database to be migrated, the admin database, and the local database |
ApsaraDB for MongoDB database | The readWrite permission on the destination database | The readWrite permission on the destination database |
To create accounts and grant permissions:
ApsaraDB for MongoDB: Manage MongoDB database users in DMSse DMS to manage MongoDB database users.
Self-managed MongoDB:
db.createUser()(official MongoDB documentation)oDB Create User documentation.
Configure and run a migration task
Step 1: Go to the Data Migration page
Via the DTS console:
Log on to the DTS console.
In the left-side navigation pane, click Data Migration.
In the upper-left corner, select the region where the migration instance resides.
Via the DMS console:
The actual steps may vary based on the DMS console mode and layout. See Simple mode and Customize the layout and style of the DMS console.
Log on to the DMS console.
In the top navigation bar, move the pointer over Data + AI > DTS (DTS) > Data Migration.
From the drop-down list to the right of Data Migration Tasks, select the region where the migration instance resides.
Step 2: Create the task and configure source and destination databases
Click Create Task.
Configure the source and destination databases using the parameters in the following tables.
After configuring the source and destination databases, review the Limits displayed at the top of the page. Skipping this step may cause the task to fail or result in data inconsistency.
Source database parameters
Parameter | Description |
Task Name | The task name. DTS generates a name automatically. Specify a descriptive name to identify the task easily. Task names do not need to be unique. |
Select Existing Connection | If the source database is registered with DTS, select it from the drop-down list. DTS populates the remaining parameters automatically. Otherwise, configure the parameters below manually. |
Database Type | Select MongoDB. |
Access Method | Select the connection type based on the location of the source database. This topic uses Public IP Address as an example. If you select a different connection type, complete the required preparations. |
Instance Region | Select the region where the source database resides. If the region is not listed, select the geographically closest region. |
Architecture | Select Replica Set. |
Migration Method | Select the method for migrating incremental data: Oplog (recommended) or ChangeStream. Oplog is available when the oplog feature is enabled on the source. ChangeStream is available when change streams are enabled. If the source is an inelastic Amazon DocumentDB cluster, you must select ChangeStream and set the Architecture parameter to Sharded Cluster. If you select Sharded Cluster for the Architecture parameter, you do not need to configure the Shard account and Shard password parameters. |
Endpoint Type | Select Standalone or Multi-node. Available when Access Method is Express Connect, VPN Gateway, or Smart Access Gateway, Public IP Address, or Cloud Enterprise Network (CEN). |
Domain Name or IP | The domain name or IP address of the source database. Available when Endpoint Type is Standalone. |
Port Number | The service port of the source database. Available when Endpoint Type is Standalone. |
Endpoint | The endpoint of the source database in |
Authentication Database | The database that the source account belongs to. Default: |
Database Account | The account used to connect to the source database. For required permissions, see Permissions required. If Access Method is Self-managed Database on ECS or Database Gateway, enter the account of the shard node. |
Database Password | The password for the database account. |
Encryption | The connection encryption method: Non-encrypted, SSL-encrypted, or Mongo Atlas SSL. Available options depend on Access Method and Architecture. SSL-encrypted is unavailable when Architecture is Sharded Cluster and Migration Method is Oplog. If Architecture is Replica Set, Access Method is not Alibaba Cloud Instance, and Encryption is SSL-encrypted, upload a CA certificate to verify the connection. |
Destination database parameters
Parameter | Description |
Select Existing Connection | If the destination instance is registered with DTS, select it from the drop-down list. Otherwise, configure the parameters below manually. |
Database Type | Select MongoDB. |
Access Method | Select Alibaba Cloud Instance. |
Instance Region | Select the region where the destination ApsaraDB for MongoDB instance resides. |
Replicate Data Across Alibaba Cloud Accounts | Select No to use an instance in the current account. |
Architecture | Select the architecture of the destination instance. |
Instance ID | Select the ID of the destination instance. |
Authentication Database | The database that the destination account belongs to. Default: |
Database Name | The name of the database in the destination instance where migrated objects will be stored. |
Database Account | The account used to connect to the destination instance. For required permissions, see Permissions required. |
Database Password | The password for the database account. |
Encryption | The connection encryption method. SSL-encrypted is unavailable when the destination uses a sharded cluster architecture. |
Step 3: Test connectivity
Click Test Connectivity and Proceed, then click Test Connectivity in the CIDR Blocks of DTS Servers dialog box.
Make sure the CIDR blocks of DTS servers are added to the allowlists of the source and destination databases. See Add DTS server IP addresses to a whitelist.
Step 4: Configure migration objects
On the Configure Objects page, configure the migration objects using the following parameters.
Parameter | Description |
Migration Types | Select the migration types: Schema Migration, Full Data Migration, and optionally Incremental Data Migration. For zero-downtime migration, select all three. If you skip Schema Migration, create the target databases and collections in the destination before starting. If you skip Incremental Data Migration, do not write to the source during migration. |
Processing Mode of Conflicting Tables | Precheck and Report Errors: fails the precheck if the destination contains collections with the same names as the source. Use object name mapping to rename migrated collections if existing collections cannot be deleted. Ignore Errors and Proceed: skips the precheck. DTS does not migrate records with the same primary keys as existing destination records. Data consistency is not guaranteed. |
Capitalization of object names in destination instance | The capitalization policy for database and collection names in the destination. Default: DTS default policy. See Capitalization policy for object names in the destination database. |
Source Objects | Select databases or collections to migrate. Click the |
Selected Objects | Right-click an object to rename it in the destination (object name mapping), set the incremental migration mode, or specify WHERE conditions for full migration data filtering. To remove an object, click it and then click the |
icon to add them to Selected Objects.
icon.Step 5: Configure advanced settings
Click Next: Advanced Settings and configure the following parameters.
Parameter | Description |
Dedicated Cluster for Task Scheduling | By default, DTS schedules tasks on a shared cluster. Purchase a dedicated cluster for isolated resources. See What is a DTS dedicated cluster? |
Retry Time for Failed Connections | The retry duration for connection failures. Valid values: 10–1,440 minutes. Default: 720 minutes. Set this to at least 30 minutes. If different tasks share the same source or destination database, the value set last takes precedence. |
Retry Time for Other Issues | The retry duration for DDL or DML operation failures. Valid values: 1–1,440 minutes. Default: 10 minutes. Set this to at least 10 minutes. This value must be less than Retry Time for Failed Connections. |
Enable Throttling for Full Data Migration | Limits read/write load on source and destination during full migration. Configure QPS to the source database, RPS of Full Data Migration, and data migration speed. Available only when Full Data Migration is selected. |
Only one data type for primary key `_id` in a table of the data to be synchronized | Yes: DTS migrates only one data type of |
Enable Throttling for Incremental Data Migration | Limits load during incremental migration. Configure RPS of Incremental Data Migration and data migration speed. Available only when Incremental Data Migration is selected. |
Environment Tag | An optional tag to identify the instance. |
Configure ETL | Enable the extract, transform, and load (ETL) feature to transform data during migration. See Configure ETL in a data migration or data synchronization task. |
Monitoring and Alerting | Configure alerts for task failure or high migration latency. If enabled, set the alert threshold and notification contacts. See Configure monitoring and alerting. |
Step 6: Configure data verification (optional)
Click Next Data Verification to set up a data verification task. See Configure a data verification task.
Step 7: Run the precheck
Click Next: Save Task Settings and Precheck.
To preview the API parameters for this task configuration, move the pointer over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.
DTS runs a precheck before starting the migration. The task starts only after passing the precheck.
If a check item fails, click View Details, resolve the issue, then click Precheck Again.
If an alert is triggered for a check item that can be ignored, click Confirm Alert Details > Ignore > OK > Precheck Again. Ignoring alerts may result in data inconsistency.
Step 8: Purchase and start the instance
Wait for Success Rate to reach 100%, then click Next: Purchase Instance.
On the Purchase Instance page, configure the following parameters.
Parameter | Description |
Resource Group | The resource group for the migration instance. Default: default resource group. See What is resource management? |
Instance Class | The instance class determines migration speed. Select based on your data volume and timing requirements. See Instance classes of data migration instances. |
Read and select the checkbox to agree to Data Transmission Service (Pay-as-you-go) Service Terms.
Click Buy and Start, then click OK in the confirmation dialog box.