Integrate data from a MongoDB replica set database to OSS

Create an OSS bucket

1. Log on to the OSS console.

2. In the left-side navigation pane, click Buckets, then click Create Bucket.

3. Configure the following parameters: For other parameters, see Create a bucket.

   Parameter	Value
   Bucket name	Enter a name for the bucket.
   Region	Select the region where the data integration task will run.
   ACL	Select Private.

4. Click OK.

5. Create a directory in the bucket:

   1. In the left-side navigation pane, click Buckets. Find and click the bucket you created.

   2. Choose Object Management > Objects.

   3. Click Create Directory.

   4. In the Create Directory panel, enter a value for Directory Name. > Note: For directory naming requirements, see Manage directories.

   5. Click OK.

Create a RAM user and grant permissions

1. On the Users page, click Create User and configure the following:

   Parameter	Value
   Logon name	Enter a logon name. For example, dts_testuser.
   Access mode	Select Using permanent AccessKey to access.

2. Click OK. Record the AccessKey ID and AccessKey Secret — you will need them when registering the OSS instance.

3. Grant permissions to the RAM user:

   1. On the Users page, find the RAM user and click Add Permissions in the Actions column.

   2. Attach the following policies:

      Policy	Required	Description
      AliyunDTSFullAccess	Required	Permissions to manage DTS
      AliyunOSSFullAccess	Required	Permissions to manage OSS
      AliyunKMSReadOnlyAccess	Optional	Read-only permission on KMS
      AliyunKMSSecretUserAccess	Optional	Permission to retrieve secrets from KMS
      AliyunKMSCryptoUserAccess	Optional	Permission to use KMS keys for cryptographic operations

      Note: As an alternative to AliyunOSSFullAccess, grant AliyunOSSReadOnlyAccess to the RAM user, then configure a bucket policy in the OSS console with Applied To set to Whole Bucket, Authorized User set to RAM User, and Authorized Operation set to Full Access or Read/Write. For details, see Configure bucket policies by using the GUI.

   3. Click Grant permissions, then click Close.

(Optional) Purchase and configure a KMS instance

Complete this section only if you need to encrypt the integrated data.

1. Log on to the KMS console and select the target region.

2. Purchase a KMS instance:

   1. In the left-side navigation pane, choose Resource > Instances.

   2. Click Create Instance and configure the following:

      Parameter	Value
      Key management type	Select Software Key Management.
      Region	Select the region of the data integration task.

      For other parameters, see Purchase a KMS instance.

   3. Click Buy Now and follow the prompts.

3. Enable the KMS instance:

   1. Return to the Instances page.

   2. On the Software Key Management tab, find the KMS instance and click Enable in the Actions column.

   3. Follow the prompts. For parameter details, see Enable the KMS instance.

   4. Wait until the instance status changes to enabled.

4. Create a customer master key (CMK):

   1. In the left-side navigation pane, choose Resource > Keys.

   2. Optional. Click the Keys tab.

   3. On the right side of Create Key, set Instance ID to the KMS instance you created.

   4. Click Create Key and configure the following:

      Parameter	Value
      KMS instance	Select the KMS instance you purchased.
      Key type	Select Symmetric Key.
      Key specifications	Select Aliyun_AES_256.

      For other parameters, see Create a CMK.

   5. Click OK.

5. Create a generic secret:

   1. In the left-side navigation pane, choose Resource > Secrets.

   2. Optional. Click the Generic Secrets tab.

   3. On the right side of Create Secret, set Instance ID to the KMS instance you created.

   4. Click Create Secret and configure the following:

      Parameter	Value
      Secret value	Select Plain Text, then enter a secret value in the Plain Text field. To generate a random value, run openssl rand 32 \| base64 in an environment with OpenSSL installed.
      CMK	Select the CMK you created.

      For other parameters, see Create a generic secret.

   5. Click OK.

6. Create an application access policy (AAP):

   1. In the left-side navigation pane, choose Application Access > AAPs.

   2. Click the Policy tab, then click Create Policy.

   3. Configure the following:

      Parameter	Value
      Scope	Select the KMS instance you created.
      RBAC permissions	Select all available options.
      Accessible resources	Select the CMK and the generic secret you created.

      For other parameters, see Create a permission policy.

   4. Click OK.

(Optional) Purchase and configure an AnalyticDB MySQL 3.0 cluster

Complete this section only if you need to store OSS metadata in an AnalyticDB MySQL 3.0 cluster.

1. Purchase an AnalyticDB MySQL 3.0 cluster. For more information, see Create a cluster.

2. Create a privileged account. For more information, see Create a database account.

3. Create a resource group for the cluster. Set Task type to Interactive and Engine to Spark. This configuration enables data querying after integration. For more information, see Create a resource group.

Register the MongoDB instance

1. Log on to the DMS console V5.0.

2. On the Home page, click the icon next to Database Instances in the left-side navigation pane.

   Alternatively, choose Data Assets > Instances in the top navigation bar. On the Instance List tab, click New.

3. In the Add Instance dialog box, click the Third-party Cloud/Self-managed tab.

   Important

   Select Third-party Cloud/Self-managed for the ApsaraDB for MongoDB instance (replica set).

4. Under NoSQL Database, select MongoDB and configure the following parameters: For other parameters, see Register a database hosted on a third-party cloud service or a self-managed database.

   Parameter	Description
   Database type	Select MongoDB.
   Instance source	Select VPC PrivateLink.
   Instance region	Select the region where the MongoDB database resides.
   VPC ID	Enter the ID of the VPC to which the MongoDB database is connected.
   Logon address	Enter the domain name or IP address of the MongoDB database.
   Port	Enter the service port of the MongoDB database.
   Database name	Enter the name of the database to which the MongoDB account belongs. The default value is admin.
   Database account	Enter the account used to log on to the MongoDB database. The account must have read permissions on the source database, the admin database, and the local database.
   Database password	Enter the password used to access the database instance.

5. Click Submit.

Register the OSS instance

1. Log on to the DMS console V5.0.

2. On the Home page, click the icon next to Database Instances in the left-side navigation pane.

   Alternatively, choose Data Assets > Instances in the top navigation bar. On the Instance List tab, click New.

3. In the Add Instance dialog box, click the Alibaba Cloud tab.

4. Under File and Log, select OSS and configure the following parameters: For other parameters, see Register an Alibaba Cloud database instance.

   Parameter	Description
   File and log storage	Select OSS.
   Instance region	Select the region of the bucket created in Step 1.
   Connection method	Select Connection String Address.
   Connection string address	DMS automatically fills in the endpoint based on the selected region. For example, selecting China (Hangzhou) sets the address to oss-cn-hangzhou.aliyuncs.com.
   Bucket	(Optional) Select the bucket created in Step 1.
   AccessKey ID	Enter the AccessKey ID of the RAM user created in Step 1.
   AccessKey Secret	Enter the AccessKey Secret of the RAM user created in Step 1.

5. Click Submit.

(Optional) Register the AnalyticDB MySQL 3.0 cluster

Complete this step only if you need to store OSS metadata in an AnalyticDB MySQL 3.0 cluster.

1. Log on to the DMS console V5.0.

2. On the Home page, click the icon next to Database Instances in the left-side navigation pane.

   Alternatively, choose Data Assets > Instances in the top navigation bar. On the Instance List tab, click New.

3. In the Add Instance dialog box, click the Alibaba Cloud tab.

4. Under Data Warehouse, select AnalyticDB MySQL 3.0 and configure the following parameters: For other parameters, see Register an Alibaba Cloud database instance.

   Parameter	Description
   Database type	Select AnalyticDB MySQL 3.0.
   Instance region	Select the region where the AnalyticDB MySQL 3.0 cluster resides.
   Connection method	In this example, select Instance ID.
   Instance ID	Select the ID of the AnalyticDB MySQL 3.0 cluster.
   Database account	Enter the privileged account of the cluster.
   Database password	Enter the password used to access the cluster.

5. Click Submit.