Migrate JSON-formatted data from MongoDB to MaxCompute-MaxCompute(MaxCompute)-阿里云帮助中心

Prerequisites

Activate MaxCompute.
Add a MaxCompute data source. For more information, see Bind MaxCompute compute resources.
Create a business flow in DataWorks. This topic uses a business flow in basic mode. For more information, see Create a business flow.

Prepare test data in MongoDB

Prepare a user account.
Create a user in the database. You can use this user to add a data source in DataWorks. In this example, run the following command.
```
db.createUser({user:"bookuser",pwd:"123456",roles:["user1"]})
```
This command creates a user named bookuser with the password 123456. The role can be any role that has permissions to access the data.

Prepare data.

Upload data to the MongoDB database. This example uses ApsaraDB for MongoDB. The network type is VPC. A public endpoint is required for the VPC to communicate with the DataWorks public resource group. The following code provides the test data.

{
    "store": {
        "book": [
            {
                "category": "reference",
                "author": "Nigel Rees",
                "title": "Sayings of the Century",
                "price": 8.95
                },
            {
                "category": "fiction",
                "author": "Evelyn Waugh",
                "title": "Sword of Honour",
                "price": 12.99
                },
            {
                "category": "fiction",
                "author": "J. R. R. Tolkien",
                "title": "The Lord of the Rings",
                "isbn": "0-395-19395-8",
                "price": 22.99
                }
                    ],
        "bicycle": {
            "color": "red",
            "price": 19.95
                }
                    },
        "expensive": 10
            }

This example uses the admin database and the userlog collection in the DMS console for MongoDB. To view the uploaded data, run the following command.
```
db.userlog.find().limit(10)
```

Use DataWorks to migrate data

Login DataWorks console.
Create a destination table in DataWorks for the migrated MongoDB data.
1. Right-click a created Workflow, Select Create Table > Table
2. In Create Table page, select the engine type, and enter Name.
3. On the table editing page, click DDL Statement.
4. In the DDL Statement dialog box, enter the DDL statement and click Generate Table Schema.
  
  Important
  The table name in the DDL statement must match the Table Name you entered on the Create Table page.
```
create table mqdata (mqdata string);
```
5. Click Commit to Production Environment.
Add a MongoDB data source. For more information, see Configure a MongoDB data source.

Create an offline synchronization node.

Go to the data analytics page. Right-click the specified workflow and choose Create Node > Data Integration > Offline synchronization.
In Create Node dialog box, enter Name, and click Confirm.
In the top navigation bar, choose icon.
In script mode, click icon.
In import Template dialog box SOURCE type, data source, target type and data source, and click confirm.

Enter the following script.

{
    "type": "job",
    "steps": [
    {
        "stepType": "mongodb",
        "parameter": {
            "datasource": "mongodb_userlog",
            "column": [
                {
                "name": "store.bicycle.color", 
                "type": "document.String" 
                }
              ],
            "collectionName": "userlog"   
            },
        "name": "Reader",
        "category": "reader"
        },
        {
            "stepType": "odps",
            "parameter": {
            "partition": "",
            "isCompress": false,
            "truncate": true,
            "datasource": "odps_source",
            "column": [
            "mqdata"  
            ],
            "emptyAsNull": false,
            "table": "mqdata"
            },
            "name": "Writer",
            "category": "writer"
            }
            ],
            "version": "2.0",
            "order": {
            "hops": [
            {
            "from": "Reader",
            "to": "Writer"
            }
            ]
            },
            "setting": {
            "errorLimit": {
            "record": ""
            },
            "speed": {
            "concurrent": 2,
            "throttle": false,
            }
            }
        }

Click icon to run the code.
You can operation Log view the results.

Verify the result

Right-click the workflow and choose new > MaxCompute > ODPS SQL.
In create a node dialog box, enter node name, and click submit.
On the ODPS SQL node editing page, enter the following statement.
```
SELECT * from mqdata;
```
Click icon to run the code.
You can Runtime Log view the results.