Public dataset data source-DataWorks(DataWorks)-阿里云帮助中心

DataWorks provides a built-in public dataset data source that lets you quickly try single-table offline data synchronization without configuration. This topic describes the public datasets and regions supported for data synchronization in DataWorks.

Supported datasets and regions

For a list of supported public datasets and their details, see the DataWorks GalleryCloud Marketplace Datasets category. You must subscribe to a dataset before you can use it in a synchronization task.
The public dataset data source is available in the following regions:
China (Beijing), China (Shanghai), China (Hangzhou), China (Shenzhen), China (Zhangjiakou), China (Chengdu), China (Ulanqab), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Germany (Frankfurt), UK (London), US (Silicon Valley), and US (Virginia).

Develop a data synchronization task

For the task configuration entry points and general configuration procedure, see the following guides.

Single-table offline synchronization task configuration guide

For the procedure, see Wizard mode configuration and Script mode configuration.
For all parameters and a sample script for script mode, see Appendix: Sample script and parameter descriptions.

Appendix: Sample script and parameter descriptions

Configure an offline task in script mode

When you configure an offline task in script mode, write the required parameters in the task script according to the standard script format. For more information, see Script mode configuration. The following section describes the data source parameters in script mode.

Sample reader script

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "public_dataset",
            "parameter": {
                "datasource": "Curated Book Dataset",
                "column": [
                    "bookid",
                    "title",
                    "authors",
                    "average_rating",
                    "isbn",
                    "isbn13",
                    "language_code",
                    "__num_pages",
                    "ratings_count",
                    "text_reviews_count",
                    "publication_date",
                    "publisher"
                ],
                "table": "good_reads_books"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "stream",
            "parameter": {
                "print": true
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "locale": "zh_CN",
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    }
}

Reader script parameters

Parameter	Description	Required	Default value
datasource	The name of the public dataset, such as Curated Book Dataset.	Yes	None
table	The name of the table to synchronize. You can find table names in the dataset details.	Yes	None
column	The fields to read from the public dataset table, separated by commas. For example, "column": ["id", "name", "age"].	Yes	None