Export all data to MaxCompute

Codeless UI (default)

Configure the following items in the codeless UI:

• Data source: Select the source and destination data sources.

• Runtime Resource: Select a resource group. The system then automatically tests the data source connectivity.

• Data Source:

  • Table: Select the source data table from the drop-down list.

  • Primary Key Range (Start): The start of the primary key range from which to read data. The value must be in JSON array format. inf_min represents negative infinity.

    For example, if the primary key consists of an int column named id and a string column named name, the sample configuration is as follows:

    Specific primary key range	Full data
    [ { "type": "int", "value": "000" }, { "type": "string", "value": "aaa" } ]	[ { "type": "inf_min" }, { "type": "inf_min" } ]

  • Primary Key Range (End): The end of the primary key range from which to read data. The value must be in JSON array format. inf_max represents positive infinity.

    For example, if the primary key consists of an int column named id and a string column named name, the sample configuration is as follows:

    Specific primary key range	Full data
    [ { "type": "int", "value": "999" }, { "type": "string", "value": "zzz" } ]	[ { "type": "inf_max" }, { "type": "inf_max" } ]

  • Splitting Configuration: Custom splitting configuration in JSON array format. In most cases, you do not need to configure this parameter (set it to []).

    If data hotspots occur in your Tablestore instance and the automatic splitting policy of Tablestore Reader is ineffective, we recommend that you use custom splitting rules. A split specifies the split points within the primary key range. You only need to configure the shard keys, not all primary keys.

• Destination: Configure the following items. You can keep the default values for other parameters or modify them as needed.

  • Project Name in Production Environment: Displays the name of the MaxCompute project associated with the destination data source.

  • Tunnel Resource Group: By default, Common transmission resources is selected, which is the free quota of MaxCompute. You can select a dedicated Tunnel resource group as needed.

  • Table: Select the destination table. You can click Generate Target Table Schema to automatically generate the destination table.

  • Partition: The synchronized data is saved in a partition for a specified date. This can be used for daily incremental synchronization.

  • Write Mode: Select whether to clear existing data or append new data.

• Destination Field Mapping: Maps the fields from the source to the destination table. The system provides a default mapping based on the source table fields, which you can modify as needed.

After you complete the configuration, click Save at the top of the page.

Code editor

To edit the script, click Code Editor at the top of the page.

The following sample script is for a source data table where the primary key consists of an int column named id and a string column named name. The attribute column is an int field named age. In your script, replace the values for the datasource and table parameters with your actual values.

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "ots",
            "parameter": {
                "datasource": "source_data",
                "column": [
                    {
                        "name": "id",
                        "type": "INTEGER"
                    },
                    {
                        "name": "name",
                        "type": "STRING"
                    },
                    {
                        "name": "age",
                        "type": "INTEGER"
                    }
                ],
                "range": {
                    "begin": [
                        {
                            "type": "inf_min"
                        },
                        {
                            "type": "inf_min"
                        }
                    ],
                    "end": [
                        {
                            "type": "inf_max"
                        },
                        {
                            "type": "inf_max"
                        }
                    ],
                    "split": []
                },
                "table": "source_table",
                "newVersion": "true"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "odps",
            "parameter": {
                "partition": "pt=${bizdate}",
                "truncate": true,
                "datasource": "target_data",
                "tunnelQuota": "default",
                "column": [
                    "id",
                    "name",
                    "age"
                ],
                "emptyAsNull": false,
                "guid": null,
                "table": "source_table",
                "consistencyCommit": false
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    },
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    }
}

After you finish editing the script, click Save at the top of the page.