Export full data to OSS

Codeless UI

Configure the following items:

• Data source: Select the source and destination data sources.

• Runtime Resource: Select a resource group. The system then automatically tests the data source connectivity.

• Data Source:

  • Table: Select the source data table from the drop-down list.

  • Primary Key Range (Start): The start of the primary key range to read, specified as a JSON array. The value inf_min represents negative infinity.

    If the primary key consists of an int primary key column named id and a string primary key column named name, see the following configuration examples:

    Primary key range	Full data
    [ { "type": "int", "value": "000" }, { "type": "string", "value": "aaa" } ]	[ { "type": "inf_min" }, { "type": "inf_min" } ]

  • Primary Key Range (End): The end of the primary key range to read, specified as a JSON array. The value inf_max represents positive infinity.

    If the primary key consists of an int primary key column named id and a string primary key column named name, see the following configuration examples:

    Primary key range	Full data
    [ { "type": "int", "value": "999" }, { "type": "string", "value": "zzz" } ]	[ { "type": "inf_max" }, { "type": "inf_max" } ]

  • Splitting Configuration: This parameter is usually not required. Set it to [].

    If hotspots in your Tablestore data render the automatic splitting policy ineffective, use custom splitting rules. These rules define split points within the primary key range. You only need to configure the split key, not every primary key column.

• Destination: Select the target Text Type and configure the related parameters.

  • Text Type: Options include csv, text, orc, and parquet.

  • Object Name (Path Included): The path and name for the destination file in the OSS bucket. Example: tablestore/resource_table.csv.

  • Column Delimiter: The default is ,. If the delimiter is a non-printable character, enter its Unicode encoding, such as \u001b or \u007c.

  • Object Path: Required only for the parquet text type. Specifies the file path within the OSS bucket.

  • File Name: Required only for the parquet text type. Specifies the file name within the OSS bucket.

• Destination Field Mapping: Maps fields from the source table to the destination file. Each line, specified in JSON format, represents one field.

  • Source Field: Includes the primary key fields and attribute columns from the source table.

    If the primary key consists of an int primary key column named id and a string primary key column named name, and there is an int attribute column named age, see the following example:

    {"name":"id","type":"int"}
    {"name":"name","type":"string"}
    {"name":"age","type":"int"}

  • Target Field: Includes the primary key fields and attribute columns from the source table.

    If the primary key consists of an int primary key column named id and a string primary key column named name, and there is an int attribute column named age, see the following example:

    {"name":"id","type":"int"}
    {"name":"name","type":"string"}
    {"name":"age","type":"int"}

After you complete the configuration, click Save at the top of the page.

Script mode

At the top of the page, click Code Editor to switch to the script editing view.

The following example script is for exporting to a CSV file. The source table's primary key consists of an int column named id and a string column named name. The table also includes an int attribute column named age. When you use the script, replace the datasource, table, and object values with your own.

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "ots",
            "parameter": {
                "datasource": "source_data",
                "column": [
                    {
                        "name": "id",
                        "type": "int"
                    },
                    {
                        "name": "name",
                        "type": "string"
                    },
                    {
                        "name": "age",
                        "type": "int"
                    }
                ],
                "range": {
                    "begin": [
                        {
                            "type": "inf_min"
                        },
                        {
                            "type": "inf_min"
                        }
                    ],
                    "end": [
                        {
                            "type": "inf_max"
                        },
                        {
                            "type": "inf_max"
                        }
                    ],
                    "split": []
                },
                "table": "source_table",
                "newVersion": "true"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "oss",
            "parameter": {
                "dateFormat": "yyyy-MM-dd HH:mm:ss",
                "datasource": "target_data",
                "writeSingleObject": false,
                "column": [
                    {
                        "name": "id",
                        "type": "int"
                    },
                    {
                        "name": "name",
                        "type": "string"
                    },
                    {
                        "name": "age",
                        "type": "int"
                    }
                ],
                "writeMode": "truncate",
                "encoding": "UTF-8",
                "fieldDelimiter": ",",
                "fileFormat": "csv",
                "object": "tablestore/source_table.csv"
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    },
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    }
}

After you finish editing the script, click Save at the top of the page.