Release notes

更新时间:
复制 MD 格式

This topic describes the release notes of DataWorks features and related documentation.

2026

April 2026

Feature

Description

Release date

Region

Target user

Documentation

DataWorks builds a personalized AI assistant based on the OpenClaw framework

DataWorks builds a personalized AI assistant for users based on the OpenClaw framework.

2026-04-30

China (Zhangjiakou), China (Beijing), China (Ulanqab), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Japan (Tokyo)

All DataWorks users

DataWorks intelligent assistant overview

Increased free quota and pay-as-you-go billing for Open API calls in Standard Edition and Professional Edition

For DataWorks Standard Edition and Professional Edition users, the daily API call limit is removed. The free quota for Standard Edition is adjusted to 100,000 calls per month, and for Professional Edition to 500,000 calls per month. Excess calls are billed on a pay-as-you-go basis for Open API. Basic Edition and Enterprise Edition remain unchanged. The QPS limits for all editions remain unchanged. For details, see the Alibaba Cloud announcement.

2026-04-23

All regions

All DataWorks users

Use DataWorks Open API

Product announcement

March 2026

Feature

Description

Release date

Region

Target user

Documentation

Full database batch synchronization from MySQL to DLF (Paimon)

Full database batch synchronization from MySQL to DLF is supported, with Paimon as the target format.

2026-03-26

All regions

All DataWorks users

/

Upgrade a workspace from basic mode to standard mode

Workspaces can be upgraded from basic mode to standard mode. Workspaces that have enabled the new Data Studio IDE are supported.

2026-03-26

All regions

All DataWorks users

/

DLF tables support permission request

DLF supports the permission request feature. You can initiate a data access permission request directly from Data Map.

2026-03-26

All regions

All DataWorks users

/

DLF tables support data preview based on Serverless Spark

DLF supports data preview based on Serverless Spark. You can use Spark compute resources in DataWorks and custom query scripts to quickly preview data.

2026-03-26

All regions

All DataWorks users

/

ChatBI provides deep analysis mode

DataWorks ChatBI introduces a deep analysis mode that supports multi-dimensional drill-down, root cause analysis, time series comparison, anomaly detection, and intelligent attribution, suitable for exploratory analysis scenarios.

2026-03-26

China (Zhangjiakou), China (Chengdu), China (Shenzhen), China (Beijing), China (Hangzhou), China (Shanghai), China (Hong Kong), Indonesia (Jakarta)

All DataWorks users

/

Data Service supports Cloud-native API Gateway

DataWorks Data Service adds support for Cloud-native API Gateway. You can use wizard mode or script mode to create data APIs and deploy them to Cloud-native API Gateway with one click for online API services.

2026-03-26

All regions

All DataWorks users

Create an API by using Cloud-native API Gateway

DataWorks supports highly available Kyuubi Gateway for Serverless Spark

If high availability is enabled for Serverless Spark that has been associated with DataWorks as a compute resource, both Serverless Spark SQL and Serverless Kyuubi tasks can be submitted to the compute resource through the highly available Kyuubi Gateway.

2026-03-26

All regions

All DataWorks users

/

Data Analysis provides full data query mode for Hologres, ClickHouse, and MySQL data sources

DataWorks Data Analysis supports full data query mode for Hologres, ClickHouse, and MySQL data sources, suitable for large data query and analysis scenarios where a single query exceeds 10,000 rows or 10 MB.

2026-03-26

All regions

All DataWorks users

/

February 2026

Feature

Description

Release date

Region

Target user

Documentation

Data Studio SQL nodes support data quality test rules

Data Studio MaxCompute SQL nodes support configuring data quality test rules to validate the development and production data tables produced by the node. This feature deeply integrates quality rule definition, testing, and SQL development workflows to address issues such as delayed quality rule configuration, late discovery of data issues, and high rule maintenance costs in traditional approaches.

2026-02-20

All regions

All DataWorks users

Configure data quality tests

Data Studio SQL nodes support deep pre-check for data governance

Data Studio SQL nodes support deep pre-check for data governance. You can define custom rules based on AI capabilities to detect and fix code issues in real time during coding, improving code quality and data security.

2026-02-20

All regions

All DataWorks users

Diagnose and govern node development issues

Operation Center Agent

DataWorks data operations Agent supports AI-powered end-to-end diagnostics, can output structured diagnostic reports, and supports executing operations such as rerun and modifying resource groups directly in the dialog (with manual confirmation required), improving data operations efficiency.

2026-02-20

China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Japan (Tokyo)

All DataWorks users

Data operations Agent

Data Studio supports offboarding transfer

Data Studio supports the offboarding transfer feature to help enterprises smoothly transfer data development assets when employees leave.

2026-02-20

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Thailand (Bangkok), Germany (Frankfurt), UK (London), US (Silicon Valley), US (Virginia)

All DataWorks users

/

Operations assistant supports batch operations

DataWorks operations assistant supports batch operations. You can manually enter or upload a file to perform batch operations on all instances in the file.

2026-02-20

China (Ulanqab)

All DataWorks users

/

2025

December 2025

Feature

Description

Release date

Region

Target user

Documentation

Data Integration capability enhancements

  • Full database real-time synchronization from PostgreSQL to Lindorm is supported, with automatic incremental synchronization after full synchronization completes.

  • Full database real-time synchronization from PostgreSQL to AnalyticDB for MySQL is supported, with automatic incremental synchronization after full synchronization completes.

  • Full database batch synchronization from Hive to DLF is supported, including one-time or periodic full and incremental synchronization.

2025-12-20

All regions

All DataWorks users

Data Map adds support for Paimon Catalog and MongoDB metadata collection

DataWorks adds Paimon Catalog metadata collection, supporting users who use non-DLF self-managed Paimon Catalog (on OSS FileSystem) to collect and manage metadata and view details. MongoDB periodic metadata collection is also supported, helping data developers quickly discover, understand, and manage collections and column structures in MongoDB, improving the visibility and governance efficiency of semi-structured data assets.

2025-12-20

All regions

All DataWorks users

Metadata collection

Open Data adds Data Quality open data views

Based on DataWorks Open Data, the quality instance view is released, supporting MaxCompute SQL queries on daily Data Quality instance status and statistical metrics for custom multi-dimensional quality analysis.

2025-12-20

All regions

DataWorks Enterprise Edition users

Open data table schema

November 2025

Feature

Description

Release date

Region

Target user

Documentation

For-each nodes support multi-concurrency

For-each nodes support configuring the execution policy as serial or parallel execution.

2025-11-27

All regions

All DataWorks users

for-each node

New Dependent node

DataWorks adds the Dependent node. You can configure complex cross-cycle dependencies through the Dependent node.

2025-11-27

All regions

All DataWorks users

Dependency check node

Notebook adds support for ADB Spark compute resources

In DataWorks Notebook, you can use Magic Commands to quickly connect to ADB Spark compute resources for PySpark data development scenarios.

2025-11-27

All regions

All DataWorks users

Advanced notebook development

Data Analysis SQL query results support export to OSS

After running an SQL query in Data Analysis to generate query results, you can quickly export the results by clicking Export > OSS for data archiving and reuse.

2025-11-27

All regions

All DataWorks users

Export and share

Data Integration supports public data sources as source

Data Integration supports public data sources as data sources, enabling single-table batch synchronization to any supported destination.

2025-11-20

All regions

All DataWorks users

Public dataset data source

Data Integration capability enhancements

  • Single-table real-time synchronization from Hologres to Lindorm wide table is supported.

  • Single-table batch write to AWS S3 is supported.

  • Full database real-time synchronization from PostgreSQL to MaxCompute DeltaTable is supported.

  • Single-table batch read from DataBricks data source is supported.

  • Single-table real-time synchronization from Kafka to Lindorm is supported.

  • Full database real-time synchronization from OceanBase to Hologres is supported.

2025-11-20

All regions

All DataWorks users

Overview

October 2025

Feature

Description

Release date

Region

Target user

Documentation

Data Studio supports connecting to local editors

Developers can use their familiar local IDEs for cloud project development, combining the advantages of personal development habits with cloud-based collaboration and version management.

2025-10-25

China (Zhangjiakou), China (Ulanqab), China (Beijing), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Chengdu), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Thailand (Bangkok), Japan (Tokyo), US (Silicon Valley), Germany (Frankfurt), UK (London), US (Virginia), China (Hong Kong), Singapore

Users of the new Data Studio

Connect to VS Code for local development

Code synchronization between the new IDE and Git

Support for automatic code synchronization to Git and merging from Git to Data Studio is completed. This feature provides comprehensive DevOps support for enterprise customers and enhances team collaboration capabilities.

2025-10-25

China (Zhangjiakou), China (Ulanqab), China (Beijing), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Chengdu), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Thailand (Bangkok), Japan (Tokyo), US (Silicon Valley), Germany (Frankfurt), UK (London), US (Virginia), China (Hong Kong), Singapore

Users of the new Data Studio

Git code synchronization and merge

New Serverless Spark/StarRocks nodes

Full OpenLake adaptation is completed, including support for OpenLake workspaces with new Serverless Spark/StarRocks nodes, providing a seamless and efficient development experience for OpenLake users.

2025-10-25

China (Zhangjiakou), China (Ulanqab), China (Beijing), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Chengdu), Malaysia (Kuala Lumpur), Indonesia (Jakarta), Thailand (Bangkok), Japan (Tokyo), US (Silicon Valley), Germany (Frankfurt), UK (London), US (Virginia), China (Hong Kong), Singapore

Users of the new Data Studio

Data development agent

The data development agent has end-to-end autonomous development capabilities. Based on requirement documents submitted by users, it can autonomously complete the entire ETL development process from task planning to final deployment, assisting with requirement analysis, code generation, workflow generation, schedule settings, and deployment, aiming to significantly improve development experience, reduce code error rates, and shorten project delivery cycles.

2025-10-25

China (Zhangjiakou), China (Beijing), China (Ulanqab), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Japan (Tokyo)

Users of the new Data Studio

Data development agent

Copilot Rule engine

The Copilot Rule engine is released, allowing users to influence Agent behavior based on team standards or personal preferences. Through flexible rule configuration, it better meets the standardization and personalization needs of enterprise scenarios.

2025-10-25

China (Zhangjiakou), China (Beijing), China (Ulanqab), China (Hangzhou), China (Shanghai), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Japan (Tokyo)

Users of the new Data Studio

Rule configuration

Multimodal dataset search

DataWorks supports tenant-level search of managed multimodal datasets, enabling unified search and viewing of datasets created and managed in DataWorks and PAI datasets. For PAI datasets with tags and indexes, semantic search on dataset details is supported.

2025-10-17

All regions

All users in the supported regions

Multimodal metadata management

New LLM node

The LLM node integrates large language models to provide intelligent data processing and analysis capabilities for developers, significantly lowering technical barriers and empowering more business users to participate in data work.

2025-10-17

China (Hangzhou), China (Shanghai), China (Beijing), China (Ulanqab), China (Shenzhen), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), and US (Virginia)

Users of the new Data Studio

Large language model node

DataWorks Open Data

DataWorks Open Data provides a centralized, unified, and easy-to-use metadata collection covering detailed information about tables, task nodes, instances, workspaces, members, and projects under the current tenant. By installing Open Data into a workspace associated with a MaxCompute compute resource, you can use MaxCompute Package view authorization to share metadata in data development and analysis, quickly obtain standardized and traceable metadata, and achieve efficient data governance and analysis.

2025-10-14

All regions

DataWorks Enterprise Edition users

September 2025

Feature

Description

Release date

Region

Target user

Documentation

Data Integration capability enhancements

  • Single-table real-time synchronization from Hologres to MaxCompute is supported.

  • Single-table batch read from Lindorm Iceberg tables is supported for data lake data export.

  • Single-table batch read from DLF is supported for data lake data export.

  • Batch synchronization from OceanBase sharded tables to MaxCompute in single-table mode is supported.

2025-09-22

All regions

All users in the supported regions

Overview

Data Integration alert rule templates

Alert templates can be configured in the Data Integration console, allowing one alert configuration to be used across multiple tasks, improving alert configuration efficiency.

2025-09-22

All regions

All users in the supported regions

Public Alert Rule

Data processing LLM supports custom models

Custom large language models can be configured for AI-assisted data processing in single-table batch synchronization in Data Integration, allowing you to use your own trained models for AI-related work.

2025-09-22

All regions

All users in the supported regions

AI-assisted processing

Data Integration processing supports embedding

The data processing feature for single-table batch synchronization in Data Integration introduces an embedding vectorization feature. It supports extracting data from heterogeneous sources such as OSS, MaxCompute, and HDFS, converting the data into vectors, and writing them to vector databases such as Milvus, Elasticsearch, and OpenSearch, as well as Hologres vector tables. This significantly simplifies ETL workflows, efficiently vectorizes knowledge, and supports RAG and other AI scenarios.

2025-09-22

All regions

All users in the supported regions

Vectorization

DataWorks resource groups support deploying and using LLM services

DataWorks LLM services provide one-stop capabilities for efficient deployment, secure communication, and convenient invocation. You can easily deploy models on DataWorks serverless resource groups and directly invoke models in data development tasks. All traffic is transmitted through private network channels established via PrivateLink, ensuring that data never leaves the network and effectively protecting data security.

2025-09-22

China (Hangzhou), China (Shanghai), China (Beijing), China (Ulanqab), China (Shenzhen), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), and US (Virginia)

All users in the supported regions

August 2025

Feature

Description

Release date

Region

Target user

Documentation

DataWorks workspaces support workspace templates

Workspace templates define the tools, resources, and features that can be used in a DataWorks workspace. You can add more types of compute resources and data sources as needed.

2025-08-18

China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen)

All users in the supported regions

Create a workspace

DataWorks OpenLake solution page supports creating OpenLake workspaces

You can quickly create workspaces for the OpenLake solution through the workspace module of DataWorks.

2025-08-18

China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen)

All users in the supported regions

Create an OpenLake workspace

EMR Serverless Spark compute resource adds support for Open LDAP and EMR Kyuubi

EMR Serverless Spark compute resource adds support for Open LDAP account mapping and Kyuubi configuration, enabling more efficient permission management and EMR Kyuubi node task development based on EMR Serverless Spark compute resources.

2025-08-14

All regions

Users of the new Data Studio

Associate EMR Serverless Spark computing resource

Notebook node adds Magic shortcut connection commands

During Notebook development, Magic commands can be used in Python cells to quickly create and start Livy services, connecting to MaxCompute Spark and EMR Serverless Spark compute resources for efficient development and debugging.

2025-08-07

All regions

Users of the new Data Studio

Basic notebook development

July 2025

Feature

Description

Release date

Region

Target user

Documentation

Data Studio node schedule settings add compute quota parameter

Data Studio supports configuring compute quotas in the schedule settings of MaxCompute SQL nodes and MaxCompute Script nodes.

2025-07-31

All regions

Users of the new Data Studio

Configure node scheduling

Data Studio adds clone and version restore features

Data Studio supports clone and version restore operations on workflows, nodes, and components for quick reuse and historical version recovery.

2025-07-31

All regions

Users of the new Data Studio

Use datasets

Data Studio supports using datasets in Shell nodes, Python nodes, Notebooks, and personal development environments.

2025-07-28

All regions

Users of the new Data Studio

Use datasets

Manage datasets

DataWorks adds dataset management, enabling version control for unstructured data such as images and documents.

2025-07-28

All regions

All users

Manage datasets

Data Security Guard

The Data Security Guard entry point is adjusted.

Before: All Products > Data Asset Governance > Data Security Guard

After: All Products > Data Asset Governance > Security Center. In the left-side navigation pane of Security Center, click Sensitive data management.

2025-07-10

China (Shanghai), China (Shenzhen), China (Beijing), China (Chengdu)

All users in the supported regions

-

Data Studio adds trigger-based workflows

Trigger-based workflows are suitable for scenarios without a fixed schedule. You can manually or trigger event-based runs of trigger-based workflows.

2025-07-07

China (Hangzhou), China (Ulanqab), China (Shenzhen)

Users of the new Data Studio

June 2025

Feature

Description

Release date

Region

Target user

Documentation

Convert pay-as-you-go resource groups to subscription

Pay-as-you-go serverless resource groups can be converted to subscription billing.

2025-06-19

All regions

All users

Use Serverless resource groups

Data synchronization

For full database real-time synchronization tasks from MySQL or PolarDB to MaxCompute, function expressions can be used to define column values in target table column assignments.

2025-06-11

All regions

All users

Use function expressions to assign values to destination table fields

PAIFlow

PAI Flow provides end-to-end machine learning workflow development capabilities. It offers the same workflow features as the Designer visual modeling of Platform for AI (PAI), and supports periodic scheduling of workflows.

2025-06-10

China (Hangzhou), China (Shanghai), China (Beijing), China (Ulanqab), China (Shenzhen), China (Hong Kong), Singapore, Indonesia (Jakarta), Japan (Tokyo), Germany (Frankfurt), US (Silicon Valley), US (Virginia)

Users of the new Data Studio

PAI Flow nodes

Smoke testing

Smoke testing is now supported to verify whether parameter substitution logic and execution results of scheduled node tasks meet expectations, preventing basic configuration errors from affecting production data.

2025-06-10

All regions

All users

Smoke testing

Operations dashboard adds manual tasks

The operations dashboard now supports viewing task execution status for manual workflows and internal task instances.

2025-06-04

All regions

All users

O&M dashboard

May 2025

Feature

Description

Release date

Region

Target user

Documentation

Personal development environment

When creating a DataWorks custom image in the personal development environment, a MaxCompute custom image is generated simultaneously.

2025-05-23

All regions

Users of the new Data Studio

Create a MaxCompute custom image

Code review

Code review aims to optimize production code through manual intervention to improve the code quality of production scheduled tasks and ensure that production code meets specifications. Based on business management requirements, you can decide whether to enforce manual code review before tasks are deployed to the production environment. Once mandatory code review is enabled, only reviewed tasks can be deployed to the production environment.

2025-05-23

All regions

Users of the new Data Studio

Code review

April 2025

Feature

Description

Release date

Region

Target user

Documentation

New DataWorks on Lindorm

Lindorm compute resources can be associated with workspaces. You can develop tasks through Lindorm Spark and Lindorm Spark SQL nodes, and manage and maintain Lindorm data lineage in Data Map.

2025-04-18

China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen)

Users of the new Data Studio

Data Integration

MongoDB data source adds support for versions 6.x and 7.x.

2025-04-14

All regions

All users

MongoDB data source

Data Studio/data development

Metadata mapping and data synchronization between MaxCompute and Hologres are now supported.

2025-04-10

All regions

All users

Data Studio/data development

DataWorks Agent is released, supporting data development tasks through natural language interaction.

2025-04-08

All regions

All users

DataWorks Agent for third-party clients

March 2025

Feature

Description

Release date

Region

Target user

Documentation

Data Integration

Milvus data source is supported.

2025-03-27

All regions

All users

Milvus

Data Integration

Full database real-time synchronization from MySQL to LogHub (SLS) is supported.

2025-03-12

All regions

All users

Real-time synchronization: Entire MySQL database to LogHub (SLS)

Operation Center

Copilot intelligent search and view saving features are supported.

2025-03-03

All regions

All users

February 2025

Feature

Description

Release date

Region

Target user

Documentation

Operation Center

Accessing the internal EMR Web UI in task execution logs through the public network via intelligent diagnostics is supported.

2025-02-26

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), US (Virginia), UAE (Dubai)

DataWorks Professional Edition and higher users

Intelligent diagnosis

Operation Center

When baselines are enabled, the ENABLE_TASK_PRIORITY parameter can be configured on Data Studio EMR nodes to control YARN queue priority in scheduling scenarios.

2025-02-14

All regions

All users

Map baseline priorities to YARN queue priorities

Data Studio/data development

Data Studio PyODPS 3 nodes support upgrading the PyODPS version.

2025-02-14

All regions

All users

Intelligent assistant (Copilot)

DataWorks Copilot supports DeepSeek models (DeepSeek-R1-671B and DeepSeek-R1-Distill-Qwen-32B).

2025-02-14

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu)

All users

Data Agent

Intelligent assistant (Copilot)

DataWorks Copilot adds support for code optimization and code testing features.

2025-02-14

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta)

All users

Data Agent

Personal development environment

  • Personal development environment instances support GPU image-based creation, suitable for deep learning frameworks such as TensorFlow and PyTorch.

  • Personal development environment instances support workspace-level automatic shutdown policies and scheduled shutdown configuration.

2025-02-12

All regions

Users of the new Data Studio

Personal development environment

Operation Center

Automatic operations adds an automatic rerun policy.

2025-02-10

All regions

All users

Automated O&M

Data Studio adds new nodes

SUB_PROCESS is a special task type that can reference another defined workflow task within a workflow. This task type is suitable for breaking down complex tasks into multiple subtasks, each of which can be independently defined and managed, improving task maintainability and reusability.

2025-02-06

All regions

Users of the new Data Studio

SUB_PROCESS node

January 2025

Feature

Description

Release date

Region

Target user

Documentation

Data source

PolarDB-O (sharded) data source is supported.

2025-01-14

All regions

All users

Sharding synchronization

Legacy resource groups support availability zone replacement

Resource disaster recovery capabilities are enhanced, supporting quicker and more convenient fault response.

2025-01-02

All regions

All users

2024

December

Feature

Description

Release date

Region

Target user

Documentation

Data Studio adds new nodes

Data Studio adds the ADB Spark SQL node for developing AnalyticDB Spark SQL tasks with periodic scheduling and integration with other jobs.

2024-12-19

All regions

Users of the new Data Studio

ADB Spark SQL node

Data Studio adds new nodes

Data Studio adds the ADB Spark node for developing AnalyticDB Spark tasks with periodic scheduling and integration with other jobs.

2024-12-19

All regions

Users of the new Data Studio

ADB Spark node

Hologres dynamic table usage

DataWorks Data Catalog integrates the Hologres dynamic table engine, providing visual tools for dynamic table management, scheduling dependencies, and task maintenance.

2024-12-13

All regions

Users of the new Data Studio

Using Hologres dynamic tables

Data source

TiDB data source is supported.

2024-12-04

All regions

All users

TiDB data source

Data Quality

Data Quality adds format matching quality check templates, allowing you to define custom regular expressions and expected data format. Built-in templates for common content validation such as email, phone number, mobile number, and ID number are included.

2024-12-02

All regions

All users

View built-in rule templates

November

Feature

Description

Release date

Region

Target user

Documentation

Data Integration

Full database real-time synchronization from ApsaraDB for OceanBase MySQL tenant to MaxCompute is supported.

2024-11-21

All regions

All users

Real-time sync of an entire OceanBase database to MaxCompute

Data Map

Data Map metadata collection supports collecting and managing AnalyticDB for Spark metadata.

2024-11-21

All regions

Users of the new Data Studio

Metadata collection

Data Map

Data insights can be created on the MaxCompute table details page in Data Map to obtain data statistics and distribution through deep data analysis and interpretation.

2024-11-21

China (Hangzhou), China (Shanghai), China (Shenzhen), China (Chengdu), China (Zhangjiakou), China (Beijing), China (Ulanqab)

All users

MaxCompute table data

Data Map

Searching StarRocks External Catalog tables is supported.

2024-11-19

All regions

All users

External catalog collection

Data Asset Governance

Data governance center is upgraded to Data Asset Governance, which can automatically discover issues across data storage, task computing, code development, and data quality dimensions based on pre-configured governance plans. Through health score quantitative assessments from multiple perspectives, it helps you efficiently achieve governance goals. It also provides business asset management, asset analysis, task resource consumption details, and cost estimation features to help you effectively track resource usage details.

2024-11-14

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), US (Virginia)

All users

Data Asset Governance

Compute resource

Associating AnalyticDB for Spark compute resources is supported.

2024-11-05

All regions

Users of the new Data Studio

Associate a computing resource

Personal development environment connects to Git

Integration between the personal development environment and Git repositories is supported for code version management and team collaboration.

2024-11-04

All regions

Users of the new Data Studio

Connect a personal development environment to Git

October

Feature

Description

Release date

Region

Target user

Documentation

Image management

DataWorks supports building custom images as persistent images. You no longer need to redeploy the image environment each time. Using the same image environment for each task node run ensures runtime environment consistency and effectively saves task execution time, computing costs, and traffic costs.

2024.10.18

China (Beijing), China (Shanghai), China (Shenzhen), China (Hangzhou), China (Hong Kong), China (Zhangjiakou), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Japan (Tokyo), Germany (Frankfurt), UK (London), US (Silicon Valley), US (Virginia)

All DataWorks users

Custom image

Serverless synchronization tasks

Data Integration adds the Serverless synchronization task type without the resource group concept. You only need to focus on the synchronization business itself.

2024.10.12

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Hong Kong), UK (London), US (Silicon Valley), US (Virginia), Japan (Tokyo), Germany (Frankfurt), Malaysia (Kuala Lumpur)

All DataWorks users

Serverless synchronization task configuration

September

Feature

Description

Release date

Region

Target user

Documentation

Data Studio

For data development and analysis scenarios in an OpenLake House environment, Data Studio provides multi-engine collaborative data development, visual workflow orchestration, and Data Catalog management capabilities. It also supports Python programming and debugging, and uses Notebook for interactive data exploration and analysis with AI integration. You only need to select Use Data Studio (New Version) when creating a workspace to get started.

2024.9.18

China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen)

Users of the new Data Studio

Data Studio (new version)

Real-time synchronization tasks

Single-table real-time ingestion from LogHub (SLS) to Data Lake Formation 2.0 is supported, writing table data in Paimon format with simple data processing in between.

2024.9.13

All regions

All DataWorks users

Real-time synchronization from a single LogHub (SLS) table to Data Lake Formation

Real-time synchronization tasks

Resource allocation for real-time synchronization tasks can be dynamically adjusted during execution without stopping the task. You can set an adjustment plan for scheduled adjustments.

2024.9.13

All regions

All DataWorks users

Dynamic scaling conditions for resource tuning

August

Feature

Description

Release date

Region

Target user

Documentation

Real-time synchronization tasks

Full database real-time synchronization from MySQL to SelectDB or Apache Doris is now supported.

2024.8.29

All regions

All DataWorks users

Real-time synchronization from a full MySQL database to SelectDB

Hologres data access control

DataWorks Security Center provides access control capabilities for Hologres data, including setting authorized identities, permission requests, and permission approvals. It also supports viewing permission request and approval records.

2024.8.22

All regions

All DataWorks users

Control access to Hologres data

Export SQL query results to DingTalk spreadsheet documents

After running an SQL query in DataWorks, you can directly export query results as DingTalk documents via the DingTalk app, avoiding data security risks associated with downloading query data as Excel files.

2024.8.14

China (Zhangjiakou)

China (Chengdu)

All DataWorks users

Export query results to DingTalk

Reduced permission requirements for changing MaxCompute data source default access identity

When the default access identity of a MaxCompute data source is another RAM user, the requirement is changed from "having the RAM AdministratorAccess permission" to "having the admin or super_administrator permission of the MaxCompute project".

2024.8.8

All regions

All DataWorks users

Associate a MaxCompute compute resource

Enterprise Edition supports Cloud SSO

DataWorks Enterprise Edition supports Cloud SSO. Through Alibaba Cloud's Cloud SSO, you can use a third-party or self-built IdP (Identity Provider) to log in to Alibaba Cloud and use DataWorks.

2024.8.8

All regions

All DataWorks users

DataWorks: Features by edition

July

Feature

Description

Release date

Region

Target user

Documentation

RAM permission policy update

AliyunDataWorksReadOnlyAccess adds the dataworks:ListUserResources permission (to view resource information owned by the user).

2024.7.10

All regions

All DataWorks users

E-MapReduce cluster registration feature update

The following features are added for E-MapReduce cluster registration in DataWorks:

  • Custom Kyuubi connection information for clusters is supported.

  • Registration of EMR Serverless Spark cluster types is supported.

2024.7.10

Currently, only the China (Zhangjiakou) region supports EMR Serverless Spark.

All DataWorks users

Data Studio adds new nodes

Data Studio adds the CDH Spark SQL node for developing CDH Spark SQL tasks with periodic scheduling and integration with other jobs.

2024.7.10

All regions

All DataWorks users

CDH Spark SQL node

June

Feature

Description

Release date

Region

Target user

Documentation

Data Integration adds new synchronization link

Data Integration supports synchronizing MySQL data to StarRocks through full database batch synchronization or full database real-time synchronization.

2024.06.28

All regions

All DataWorks users

Data Studio supports Data Push nodes

Data Push nodes allow you to produce data tables through upstream data processing tasks in a workflow, and then attach a Data Push node downstream to push the upstream data results to DingTalk groups or Feishu groups via message cards on a scheduled basis.

Note

To use Data Push nodes, submit a ticket to contact technical support to upgrade your scheduling resource group.

2024.6.28

  • China (Hangzhou)

  • China (Shanghai)

  • China (Beijing)

  • China (Zhangjiakou)

  • China (Shenzhen)

  • China (Chengdu)

  • China (Hong Kong)

  • Singapore

  • Malaysia (Kuala Lumpur)

  • US (Silicon Valley)

  • US (Virginia)

  • Germany (Frankfurt)

All DataWorks users

Configure a data push node

DataWorks serverless resource group released

To enhance resource management across DataWorks features and unify user experience, DataWorks launches the serverless resource group that covers the core features of the original exclusive resource group for scheduling, exclusive resource group for Data Integration, and exclusive resource group for Data Service. A single resource group can handle data synchronization, task scheduling, and API service invocation and management.

2024.6.11

  • China (Beijing)

  • China (Shanghai)

  • China (Shenzhen)

  • China (Hangzhou)

  • China (Hong Kong)

  • China (Zhangjiakou)

  • Singapore

  • Malaysia (Kuala Lumpur)

  • Indonesia (Jakarta)

  • Japan (Tokyo)

  • Germany (Frankfurt)

  • UK (London)

  • US (Silicon Valley)

  • US (Virginia)

All DataWorks users

New best practice: Task development using Lindorm compute engine

Lindorm compute engine is compatible with CDH (Cloudera's Distribution Including Apache Hadoop). You can register a CDH cluster and configure Lindorm compute engine connection information in DataWorks to perform interactive SQL queries, SQL task development, and JAR task execution based on the Lindorm compute engine.

2024.6.5

All regions

All DataWorks users

Develop tasks based on LDPS

Data Integration adds new data source

Data Integration adds support for Azure Blob Storage data source.

2024.6.3

All regions

All DataWorks users

Azure Blob Storage data source

May

Feature

Description

Release date

Region

Target user

Documentation

MySQL data source supports OSS Binlog

Data Integration MySQL data source adds support for OSS Binlog.

When you add a MySQL data source, if you set Configuration Mode to ApsaraDB for RDS and your RDS for MySQL instance is in the same region as your DataWorks workspace, you can enable Read binlogs from OSS. With this feature enabled, DataWorks fetches binlogs from OSS if direct access to the RDS for MySQL binlogs fails, preventing interruptions to real-time synchronization tasks.

2024.5.24

All regions

All DataWorks users

MySQL data source

Data Quality module redesigned

Data Quality is redesigned. Based on quality monitoring with specific monitoring rules, it validates data ranges of tables and optimizes the data quality monitoring process.

2024.5.21

The new Data Quality is being gradually rolled out. For specific supported regions, check the console. If the new Data Quality features are not available in your region, see the legacy Data Quality documentation.

All DataWorks users

Data Quality

Data Integration adds new synchronization link

Data Integration supports synchronizing Hologres data to other Hologres instances through full database batch synchronization.

2024.5.20

All regions

All DataWorks users

Batch synchronization from a full Hologres database to Hologres

DataWorks supports remote script execution on servers

DataWorks SSH nodes can remotely access the host connected to the specified SSH data source and trigger script execution on the remote host.

2024.5.15

All regions

All DataWorks users

Data Studio adds EMR Kyuubi node

The DataWorks EMR Kyuubi node enables Kyuubi task development with periodic scheduling and integration with other jobs.

2024.5.11

All regions

All DataWorks users

EMR Kyuubi node

Data Studio adds multiple database type nodes

DataWorks Data Studio supports DRDS, PolarDB MySQL, Doris, and other database node types for task development with periodic scheduling and integration with other jobs.

2024.5.11

All regions

All DataWorks users

April

Feature

Description

Release date

Region

Target user

Documentation

PostgreSQL data source configuration supports SSL authentication

SSL authentication can be configured when adding a PostgreSQL data source in Data Integration.

2024.4.26

All regions

All DataWorks users

PostgreSQL

Data governance center supports Hologres data source

Data governance center adds support for Hologres data source.

Before you can use a Hologres data source in Data Governance Center, you must collect metadata of Hologres in Data Map. For more information, see Metadata collection.

2024.4.24

Hologres data sources support Data Governance Center only in the following regions: China (Beijing), China (Shanghai), China (Hangzhou), and China (Shenzhen).

All DataWorks users

Overview

Data governance center adds materialized view feature

If you frequently run big data computing tasks with many similar subqueries, DataWorks offers an intelligent, automated solution. The automated governance feature recommends materialized views by automatically identifying and classifying similar subqueries in MaxCompute. You can then create a materialized view with a single click to significantly improve query performance and reduce resource consumption.

2024.4.12

All regions

All DataWorks users

Automated governance of materialized views

DataWorks on CDH/CDP supports OPEN LDAP authentication

When registering CDH/CDP clusters in DataWorks, custom mapping between Alibaba Cloud accounts and CDH/CDP cluster OPEN LDAP accounts is supported. After mapping, CDH/CDP tasks submitted by an Alibaba Cloud account in DataWorks are executed by the mapped OPEN LDAP account. This method is used to isolate data access permissions for different cloud accounts within CDH clusters.

2024.4.8

China (Beijing), China (Shanghai), China (Hangzhou), China (Shenzhen), China (Zhangjiakou), China (Chengdu)

All DataWorks users

Set up cluster identity mapping

March

Feature

Description

Release date

Region

Target user

Documentation

New backfill data feature released

After a scheduled task is developed, submitted, and deployed, it runs on schedule based on the schedule settings. If you want to run a scheduled task for a specific time period, you can use the backfill data feature. Backfilling writes data to corresponding time partitions by backfilling historical or future data. The new backfill data feature supports the following methods:

2024.3.28

All regions

All DataWorks users

Manage data backfill instances

Open Platform supports deploying Extensions through Function Compute

In DataWorks Extensions, you can define custom checkpoint logic for event messages to supervise user operations (for example, intercepting and blocking inappropriate behavior). By deploying Extensions through Function Compute, specific event messages are directly delivered to the corresponding Function Compute service. This approach:

  • Is simple to operate. You only need a single function to deploy an extension.

  • Incurs corresponding Function Compute fees. For details, see Function Compute billing.

  • Deployed Extensions currently only support processing pre-data download events.

2024.3.19

  • China (Beijing)

  • China (Hangzhou)

  • China (Shanghai)

  • China (Zhangjiakou)

  • China (Shenzhen)

  • China (Chengdu)

  • US (Silicon Valley)

  • US (Virginia)

  • Germany (Frankfurt)

  • Japan (Tokyo)

  • China (Hong Kong)

  • Singapore

DataWorks Enterprise Edition users

Develop and deploy extensions: Function Compute

Data Modeling supports custom model deployment policies

Data Modeling supports custom model deployment policies. After a policy is enabled, you can select the corresponding deployment mode based on the policy configuration when deploying models.

2024.3.12

All regions

DataWorks users who purchased the intelligent Data Modeling product

Publishing policy management

February

Feature

Description

Release date

Region

Target user

Documentation

Billing and cost supports bill data subscription and query analysis

Billing and cost supports subscribing to different types of bill data, such as billing item detail bills and daily aggregated billing item bills. After subscription, bill data is periodically synchronized to MaxCompute. You can use the Data Analysis feature of DataWorks to query and analyze bill data, generate visual chart cards and reports, and share your Alibaba Cloud consumption analysis reports with other users.

2024.2.22

All regions

All DataWorks users

Subscribe, query, and analyze billing data

New introduction for using CDP/CDH in DataWorks

This topic describes the basic development workflow, billing, environment preparation, and access control for using CDP/CDH in DataWorks.

2024.2.21

All regions

All DataWorks users

DataWorks on CDP/CDH

Data Service supports instance-mode StarRocks data source

After activating an E-MapReduce Serverless StarRocks cluster, you can add a StarRocks data source in instance mode in DataWorks. DataWorks Data Service supports quickly creating data APIs for StarRocks data sources to enable data sharing and openness.

2024.2.20

All regions

All DataWorks users

DataWorks on EMR Serverless StarRocks best practice

Data Map adds code search for data development

DataWorks Data Map adds a code search feature for data development, supporting cross-workspace global code search. You can precisely locate target code by keyword, improving development efficiency and reducing project redundancy.

2024.2.20

All regions

DataWorks Standard Edition and higher users

Metadata retrieval

New Data Upload & Download feature

DataWorks Data Upload & Download supports uploading local CSV files and OSS file data to MaxCompute for processing and analysis. It also manages the list of uploaded files and files downloaded from DataWorks Data Analysis and other modules for convenient data management and operations.

2024.2.20

All regions

All DataWorks users

Data Studio adds CDH cluster nodes

You can develop and periodically schedule CDH-related Hive, Spark, MR, Presto, Impala, and other tasks in DataWorks.

2024.2.19

All regions

All DataWorks users

Data Security Guard new system configuration page released

Through system configuration, you can:

  • Set the identification content and identification scope for Data Security Guard.

  • Set the watermark file retention period.

  • Set whether to display the data security level for risk identification.

  • Set the email and WebHook address for receiving alert notifications.

This helps you promptly identify and handle potential security risks.

2024.2.6

All regions

All DataWorks users

System configuration

January

Feature

Description

Release date

Region

Target user

Documentation

Data Studio and Data Analysis query results support masked display

Data Security Guard supports data classification, sensitive data identification, and masked data display for E-MapReduce table data.

When you execute SQL statements to query data in Data Studio and Data Analysis, if the data is identified as sensitive, the platform masks it with obfuscation, encryption, or other patterns according to masking rules, enhancing enterprise data security management.

2024.1.25

All regions

All DataWorks users

Data Map supports displaying real-time synchronization link data lineage

Data Map adds parsing and displaying data lineage for the following real-time synchronization links:

  • MySQL real-time write to MaxCompute/Hologres

  • Kafka real-time write to MaxCompute/Hologres

  • LogHub real-time write to MaxCompute/Hologres

  • PolarDB real-time write to MaxCompute

Combined analysis of real-time synchronization lineage and batch synchronization lineage helps you gain a more comprehensive understanding of data flow.

2024.1.15

All regions

All DataWorks users

Data lineage analysis

2023

December

Feature

Description

Release date

Region

Target user

Documentation

Data Studio (DataStudio) associates data sources

To perform data modeling, data development, or use Operation Center for periodic task scheduling in DataWorks, you must first associate created data sources or clusters with Data Studio (DataStudio). After association, you can read data from data sources or clusters and perform related development operations.

2023.12.29

All regions

All DataWorks users

Preparations before data development: Associate a data source or a cluster with DataStudio

Compute resource released

To provide a more unified product experience, DataWorks merges MaxCompute, Hologres, AnalyticDB for PostgreSQL, AnalyticDB for MySQL, and ClickHouse compute engines into compute resources, and merges E-MapReduce (EMR) and CDH/CDP engines into open-source cluster management. After the merge, operations on the original compute engines (such as creating and editing) are performed through the compute resources/open-source cluster interfaces.

2023.12.29

All regions

All DataWorks users

New extension point events

  • DeleteProject: Pre-event for deleting a project.

  • ProjectDeleted: Post-event for deleting a project.

  • DownloadResources: Data download event.

2023.12.27

All regions

All DataWorks users

New extension point scope

Extension point events add scope:

  • Tenant level: Tenant-level events apply to the entire tenant.

  • Workspace level: Workspace-level events apply only to the target workspace.

When registering an extension, you can only select extension point events of one type.

2023.12.22

  • China (Beijing)

  • China (Hangzhou)

  • China (Shanghai)

  • China (Zhangjiakou)

  • China (Shenzhen)

  • China (Chengdu)

  • US (Silicon Valley)

  • US (Virginia)

  • Germany (Frankfurt)

  • Japan (Tokyo)

  • China (Hong Kong)

  • Singapore

All DataWorks users

Data governance center adds SQL efficiency optimization check items

Data governance center adds 5 check items for ODPS, Hive, and Spark SQL including Cartesian product checks, invalid full table JOIN checks, and brute force scans, helping you perform proactive pre-checks and timely optimizations during development, improving computing efficiency, avoiding significant computing resource waste, and ensuring timely data delivery.

2023.12.22

All regions

All DataWorks users

Configure check items

New StarRocks data source adaptation

DataWorks fully adapts to StarRocks data source:

  • Data Integration: Supports synchronizing StarRocks data.

  • Data Studio: Supports creating StarRocks tasks with periodic scheduling.

  • Data Analysis: Supports querying and analyzing StarRocks data.

  • Data Service: Supports creating APIs from StarRocks data tables.

  • Data Map: Supports managing StarRocks metadata with search and display capabilities.

2023.12.15

All regions

All DataWorks users

New E-MapReduce Hadoop cluster version adaptations

DataWorks adds adaptations for over ten E-MapReduce Hadoop cluster versions, including:

  • EMR-3.26.3

  • EMR-3.27.2

  • EMR-3.29.0

  • EMR-3.32.0

  • EMR-3.35.0

  • EMR-3.38.2

  • EMR-3.38.3

  • EMR-4.3.0

  • EMR-4.4.1

  • EMR-4.5.0

  • EMR-4.5.1

  • EMR-4.6.0

  • EMR-4.8.0

  • EMR-4.9.0

  • EMR-5.2.1

  • EMR-5.4.3

  • EMR-5.6.0

2023.12.15

All regions

All DataWorks users

EMR development guidelines

Data Studio Check node adds new check objects

Check nodes can verify whether target objects (MaxCompute partitioned tables, FTP files, or OSS files) are available. Once the check policy is met, the Check node returns a success status.

If a task depends on a target object, you can use a Check node to verify the target object and set the task as a downstream task of the Check node. Once the check policy is met, the Check node succeeds and triggers the downstream task.

2023.12.08

All regions

All DataWorks users

Check node

Data Studio adds PAI DLC node

You can use PAI DLC nodes to periodically schedule and run PAI DLC tasks.

2023.12.08

All regions

All DataWorks users

Create and use a PAI DLC node

Security Center adds real-time risk identification rules

Security Center allows administrators to register risk identification capabilities as Extensions in DataWorks, which serve as risk identification rules to identify risks in operational behaviors.

Currently, pre-set/custom Extensions support risk identification for data download operations, with options to request download approval or block download behavior as needed.

2023.12.08

All regions

All DataWorks users

Fraud Detection rules

November

Feature

Description

Release date

Region

Target user

Documentation

Data Studio (DataStudio) adds Check node

The DataWorks Check node can verify whether a MaxCompute partitioned table is available (i.e., the target partition exists or partition data write is confirmed complete). When a downstream task depends on the MaxCompute partitioned table, you can use this feature to verify whether the partition data is available before proceeding, avoiding operations on incorrect data.

2023.11.20

  • China (Chengdu)

  • China (Zhangjiakou)

  • China (Beijing)

  • China (Shanghai)

  • Malaysia (Kuala Lumpur)

All DataWorks users

Check node

Exclusive resource group for Data Service available in new regions

The exclusive resource group for Data Service is available in the following regions: China (Zhangjiakou), China (Hong Kong), and Singapore, along with the corresponding specifications and billing standards.

2023.11.29

All regions

All DataWorks users

October

Feature

Description

Release date

Region

Target user

Documentation

Copilot preview release

DataWorks Copilot can quickly perform various SQL-related operations based on natural language (such as SQL generation, SQL rewriting, SQL correction, and SQL comments), helping you easily and efficiently complete data ETL and data analysis work, saving significant time and effort.

2023.10.30

  • China (Chengdu)

  • China (Zhangjiakou)

  • China (Beijing)

  • China (Shanghai)

  • China (Shenzhen)

  • China (Hangzhou)

  • China (Ulanqab)

Users who registered for the preview

Use Copilot and enhanced analysis for data query and visualization

Data governance center adds governance plans

Governance plans help data governance managers plan governance work in a scenario-based manner, enabling multi-person collaborative governance with clear goals, easy execution, and quantifiable results.

The product provides multiple governance plan templates, such as resource cost optimization and data stability governance, with preset governance goals for quickly selecting assets to be governed and automatic periodic evaluation of governance effectiveness.

2023.10.25

  • China (Chengdu)

  • China (Zhangjiakou)

  • China (Beijing)

  • China (Shanghai)

  • China (Shenzhen)

  • China (Hangzhou)

  • China (Ulanqab)

DataWorks Enterprise Edition and higher users

Develop a data governance plan

August

Feature

Description

Release date

Region

Target user

Documentation

DataWorks supports custom schedules

Building on existing schedule types, a scheduling calendar is added, allowing you to mark scheduling days or non-scheduling days on the calendar to achieve custom scheduling cycles.

2023.08.24

All regions

DataWorks Enterprise Edition users

Configure scheduling calendars

Data governance center supports E-MapReduce data lake development governance

DataWorks data governance center adds proactive governance analysis based on the E-MapReduce cluster + DLF + DataWorks data lake development pipeline, including:

  • Support for governance health score evaluation.

  • Support for automatic issue detection across development and storage governance dimensions.

  • Support for pre-check issue prevention based on Hive SQL and Spark SQL.

2023.08.24

  • China North 2 Ali Gov 1

  • China East 2 Finance

  • China (Shanghai)

  • China (Hangzhou)

  • China (Beijing)

  • China (Shenzhen)

  • China (Chengdu)

  • China (Hong Kong)

  • Singapore

  • US (Silicon Valley)

  • Germany (Frankfurt)

  • Indonesia (Jakarta)

DataWorks Enterprise Edition and higher users

Overview

New visual enhanced analysis capabilities released

Data analysis > SQL Query supports the following operations through the enhanced analysis feature on analysis results:

  • Visualize results and save as data cards to generate data reports.

  • Share analysis results.

This significantly improves enterprise data analysis efficiency.

2023.08.15

  • China (Shanghai)

  • China (Hangzhou)

  • China (Beijing)

  • China (Shenzhen)

  • China (Zhangjiakou)

  • China (Chengdu)

  • China (Ulanqab)

All DataWorks users

Visualization cards and reports

June

Feature

Description

Release date

Region

Target user

Documentation

Data Modeling releases industry model templates

Based on Alibaba Cloud model building project experience, free retail industry model templates and paid automotive industry model templates are provided, including data warehouse layering and domain division, data standards, model tables, data metrics, and other rich examples to help enterprises quickly establish standardized data warehouse construction.

2023.06.01

All regions

All DataWorks users

Industry model templates

Kafka data source real-time synchronization to Hologres with ETL

  • Real-time synchronization from Kafka to Hologres supports JSON semantic parsing and other basic data processing during synchronization, implementing an ETL pipeline within real-time synchronization.

  • Supports extracting specified JSON path keys and values, as well as dynamic key and value expansion, suitable for scenarios where Kafka source message formats change.

  • Supports simulated runs during task configuration to verify data processing correctness before writing results.

2023.06.01

All regions

All DataWorks users

Kafka data source

Full database real-time synchronization from MySQL to OSS data lake in Hudi format

Full database real-time synchronization from MySQL to OSS data lake, writing in Hudi format.

  • Supports automatic integration with Alibaba Cloud DLF for metadata management.

  • Supports instance-level synchronization, allowing simultaneous selection of multiple databases from the source MySQL.

  • Supports selecting source MySQL databases and tables by regular expressions.

  • Supports automatic database and table addition, where new databases or tables added to the MySQL data source are automatically synchronized to OSS without manual intervention.

2023.06.01

All regions

All DataWorks users

OSS

Data Integration supports AWS RDS data source

AWS RDS is configured as a MySQL data source. All capabilities of the MySQL data source also apply to AWS RDS.

2023.06.01

All regions

All DataWorks users

MySQL data source

April

Feature

Description

Release date

Region

Target user

Documentation

Save Data Analysis results directly as MaxCompute tables

Data Analysis results can be saved directly as MaxCompute tables without creating tables through code, for subsequent queries or combined analysis.

2023.4.20

All regions

All DataWorks users

SQL query (legacy)

Data Analysis supports downloading up to millions of query results

The default download limit for Data Analysis SQL queries is 10,000 rows. For DataWorks Standard, Professional, Enterprise, and Flagship editions, administrators can set the maximum download limit in Security Center: Standard (200,000), Professional (2,000,000), Enterprise (5,000,000), and Flagship (5,000,000). The download feature can also be disabled.

2023.4.18

All regions

All DataWorks users

SQL query (legacy)

Big data public datasets released

Using big data AI public datasets (Taobao, Fliggy, Ali Music, GitHub, TPC, etc.), you can quickly perform TB-level big data analysis through DataWorks and MaxCompute.

2023.4.11

All regions

All DataWorks users

SQL query (legacy)

March

Feature

Description

Release date

Region

Target user

Documentation

Data governance center supports notification-based governance issue management

Administrators and individual users can send daily governance issues to be resolved from the governance workbench as notifications via system prompts, emails, DingTalk groups, and Webhooks to specified recipients, enabling governance owners to promptly track governance progress and quickly access the system for issue resolution.

2023.3.15

All regions

All DataWorks users

Configure a periodic notification for governance issues

Data governance center adds storage long lifecycle governance item

The storage long lifecycle governance item helps you set appropriate lifecycles for MaxCompute partitioned tables, reducing storage resource waste.

2023.3.15

All regions

All DataWorks users

Handle governance issues

Data Service acceleration service commercialized

The acceleration service generates online API services for MaxCompute data sources, providing good query performance and efficiency for online query requirements without exporting MaxCompute data.

2023.3.1

China (Shanghai), China (Beijing), China (Hangzhou), and China (Shenzhen).

All DataWorks users

Acceleration service

January

Feature

Description

Release date

Region

Target user

Documentation

DataWorks adds purchased resource management

The following features are added:

DataWorks displays all currently purchased and unreleased products, enabling you to quickly upgrade, downgrade, refund, or renew subscriptions.

2023.1.11

All regions

All DataWorks users

Overview

DataWorks data governance center adds batch graceful task undeploy

The following features are added:

  • Provides scenario-based special governance plans, primarily for batch safe undeployment of invalid or duplicate tasks.

  • Provides the Shutdown governance plan. Governance owners can quickly confirm affected personnel and assets by selecting governance targets.

  • Provides step-by-step execution for collaborative completion of node pausing, delaying, undeploying, and stage status notifications, ensuring orderly and smooth task undeployment.

2023.1.9

All regions

All DataWorks users

Graceful shutdown

DataStudio basic mode supports code review

The following features are added:

Basic mode supports code review. After enabling mandatory code review, node tasks take effect in the production environment only after review approval.

2023.1.5

All regions

All DataWorks users

Code review

2022

November

Feature

Description

Release date

Region

Target user

Documentation

DataWorks Data Service supports creating APIs for development and production environment data sources

In a standard mode workspace, DataWorks adds the following features:

  • Supports setting API advanced parameters based on the selected data source environment type (development environment or production environment).

  • Enables testing with development environment databases and deploying APIs with production environment databases.

2022.11.29

All regions

All DataWorks users

Create an API from a data source (API Gateway)

DataWorks Data Map supports requesting Hive table permissions

DataWorks Data Map > EMR Hive Table Details page adds the Requested Permissions button, supporting navigation to Security Center for table permission requests.

2022.11.29

All regions

All DataWorks users

Hive tables supporting data preview

DataWorks Data Map adds Data Collection feature

DataWorks Data Map adds the Data Album page with the following features:

  • Supports classifying, grading, organizing, and managing data tables from a business perspective.

  • Supports adding frequently used, team, or popular tables to target collections for quick and convenient search and retrieval.

2022.11.16

All regions

All DataWorks users

Data Albums

DataWorks Data Analysis upgrade with a brand-new SQL query experience

DataWorks Data Analysis adds the following features:

  • Supports unified management of all personal SQL files and commonly used data table collections for data retrieval.

  • Supports business data extraction through SQL statements for authorized users.

  • Supports secondary processing and visual chart display of SQL query results.

2022.11.15

All regions

All DataWorks users

SQL query (legacy)

DataWorks Data Service supports parsing request and response parameters in advanced SQL

In Data Service, the following feature is added:

  • When creating an API through the advanced script mode, one-click parameter parsing is supported in the Request Parameters and Response Parameters panels in the right-side navigation pane.

  • Reduces the burden of manually entering parameters.

2022.11.10

All regions

All DataWorks users

N/A

October

Feature

Description

Release date

Region

Target user

Documentation

DataWorks intelligent Data Modeling supports E-MapReduce Hive with modeling capabilities on par with MaxCompute

DataWorks Data Modeling > Dimensional Modeling adds the following two features:

  • Supports deploying models to E-MapReduce Hive and generating corresponding ETL framework code.

  • Supports reverse engineering existing E-MapReduce Hive physical tables into models.

2022.11.25

All regions

All DataWorks users

DataWorks intelligent Data Modeling supports model version management

DataWorks Data Modeling > Dimensional Modeling adds the following two features:

  • Supports model version management. Only formally submitted models can be deployed.

  • Supports comparing and rolling back different versions of the same model.

2022.11.25

All regions

All DataWorks users

Materialize a logical model

DataWorks Data Service API details page displays call addresses for each domain

The API details page displays the call addresses generated for the current API based on public domain, VPC domain, and independent domain. You can flexibly use different domains for API calls.

2022.10.21

All regions

All DataWorks users

View the details of an API

DataWorks Data Map lineage diagram fully upgraded

The new data lineage information provides a better lineage analysis experience. On the lineage details page, you can:

  • View upstream and downstream nodes of a table or table column.

  • View the original data source and final data destination of a table.

  • Perform impact analysis on lineage at different levels.

2022.10.21

All regions

All DataWorks users

View the details of a table

DataWorks data governance center adds 4 development dimension check items

DataWorks data governance center adds 4 check items:

  • JOIN condition type consistency.

  • Prohibiting use of certain assets.

  • Same-name UDF usage.

  • Development environment write restrictions.

Key features include:

  • Management and check capabilities.

    You can enable and configure the new check items in configuration management and view governance knowledge base usage details.

  • Proactive development data governance is performed during task submission and deployment in data development.

2022.10.20

All regions

All DataWorks users

Configure check items

DataStudio basic mode supports code review

When mandatory code review is enabled in basic mode, the code takes effect in the production environment only after review is completed.

2022.9.22

All regions

All DataWorks users

Code review

August

Feature

Description

Release date

Region

Target user

Documentation

Operation Center adds workflow-based task management

DataWorks Operation Center supports viewing task execution status from the workflow perspective and performing workflow operations such as rerun, freeze, and terminate.

2022.8.22

All regions

All DataWorks users

Auto triggered instances

Data Service adds accelerated query for MaxCompute data source

DataWorks Data Service supports generating online APIs for MaxCompute data sources. Without exporting MaxCompute data, it provides good query performance and efficiency to meet online query requirements. Acceleration options include:

  • Acceleration service based on Hologres external tables.

  • Acceleration service based on MCQA.

2022.8.17

China (Shanghai), China (Shenzhen)

All DataWorks users

Acceleration service

Data Service adds intelligent diagnostics and call chain analysis

DataWorks Data Service supports analyzing API call logs. You can use this feature to analyze the call chain of individual requests, promptly detect anomalous requests, and quickly identify issues with diagnostic results and recommendations.

2022.8.7

All regions

DataWorks users

View and analyze API call logs (public preview)

Data Map adds fine-grained permission control at project and table levels

DataWorks Data Map supports various granularity levels of metadata permission control through diverse configurations, including:

  • Whether to allow other project members to view the current project's metadata in Data Map.

  • Whether to allow non-project members, non-table owners, and non-workspace administrators to view table metadata in Data Map.

2022.8.5

All regions

DataWorks users

Data Map permission management

Data Integration supports wizard mode batch synchronization for DM Database

DataWorks Data Integration supports using the visual wizard mode for batch synchronization of DM Database data, which is easier to operate compared to script mode.

2022.8.2

All regions

DataWorks users

Codeless UI configuration

July

Feature

Description

Release date

Region

Target user

Documentation

DataWorks intelligent Data Modeling supports dimensional modeling

DataWorks intelligent Data Modeling supports the following features:

  • You can directly reference columns and partition information from existing tables or views in Hologres during model design as columns for the current model.

  • You can perform one-click filling for columns with empty display names or descriptions during model design.

    In most cases, physical tables have column descriptions. If there are no column display names, this feature allows you to quickly supplement display name information, improving modeling efficiency.

  • You can create new DataStudio nodes or associate existing DataStudio nodes during model development, improving model ETL development efficiency.

2022.7.29

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, China East 2 Finance, China South 1 Finance, China North 2 Ali Gov 1, Germany (Frankfurt), US (Silicon Valley)

All DataWorks users

DataWorks intelligent Data Modeling supports viewing associated table information

You can open the edit page of a derived metric or atomic metric, and view the associated model table name and column name in the right-side navigation pane, with navigation to the target table edit page for managing associations.

2022.7.29

All DataWorks users

Derived metrics

Intelligent Data Modeling supports setting naming rule checkers for models and metrics

Under the data warehouse layering in data warehouse planning, you can set naming checkers for model or metric types. When designing models and metrics, the naming checker validates entity names to improve naming standardization throughout the development process.

Naming rules:

  • Rule strength: A strong rule indicates recommended filling and mandatory checks on names, while a weak rule indicates recommended filling only.

  • Rule definition: The organization order and composition of names are set based on multiple elements.

2022.7.29

All DataWorks users

Data warehouse layering

DataWorks Data Analysis supports configuring exclusive resource groups

The primary account can configure Data Analysis in system settings. You can run SQL queries on a specified exclusive resource group.

2022.7.29

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), Germany (Frankfurt)

All DataWorks users

System management

Data Integration DataStudio intelligence

Supports Postgres database. SSL authentication supports dual-file mode with .crt and .key files.

2022.7.26

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Germany (Frankfurt), UK (London), US (Silicon Valley), US (Virginia), UAE (Dubai)

All DataWorks users

Data Integration

DataWorks supports EMR new data lake DataLake

DataWorks supports the new EMR data lake DataLake based on the EMR compute engine, enabling data integration, data modeling, data development and scheduling, data quality, data map, data security, data analysis (exclusive resource groups), data service, and other full data lifecycle capabilities based on the EMR engine.

2022.7.8

China (Chengdu), China (Zhangjiakou), China (Shenzhen), US (Silicon Valley), China (Beijing), China (Shanghai), Japan (Tokyo), Germany (Frankfurt), US (Virginia), Indonesia (Jakarta), UK (London), China (Hangzhou), Singapore, China (Hong Kong), Malaysia (Kuala Lumpur), UAE (Dubai)

All DataWorks users

EMR development guidelines

DataStudio intelligent editor supports visual column insertion and table permission verification

  • The intelligent editor automatically identifies the column query portions of the code. You can hover over a table name, select the columns to query, and click confirm. The column names are automatically inserted into the code.

  • The intelligent editor provides table permission verification. You can directly apply for permissions based on prompts.

2022.7.2

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong)

All DataWorks users

DataStudio (legacy)

June

Feature

Description

Release date

Region

Target user

Documentation

Data governance center fully available

Data governance center features include:

  • Quantitatively evaluates governance health scores across five dimensions: storage, computing, development, quality, and security, and automatically discovers and prevents various data governance issues.

  • Provides task resource consumption details, resource consumption trends, single-task cost estimation, and other rich features to help you effectively optimize and control resource costs.

Note
  • Data governance center became fully available on July 5, 2022, with a one-month limited-time trial.

  • After August 5, 2022, all capabilities are available in DataWorks Enterprise Edition.

2022.6.27

China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Chengdu), Singapore, US (Silicon Valley)

All DataWorks users

Overview

Data governance center supports task 360 feature

The task 360 panoramic feature centrally displays key information associated with a task, including governance item issues, change event records, affected baselines, and task execution information, enabling you to perform scheduling task governance based on this information.

2022.6.24

China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Chengdu), Singapore, US (Silicon Valley)

All DataWorks users

Task 360 overview

Intelligent Data Modeling supports finding and creating views

  • You can directly reference existing view columns and partition information as columns for the current model during model design.

  • You can materialize completed models as views.

2022.6.22

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, US (Silicon Valley), Germany (Frankfurt), China East 2 Finance, China South 1 Finance, China North 2 Ali Gov 1

All DataWorks users

Materialize a logical model

Intelligent Data Modeling supports reverse engineering models by table name keywords

During reverse modeling, you can fuzzy match by table name keywords to reverse engineer logical models.

2022.6.19

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, US (Silicon Valley), Germany (Frankfurt), China East 2 Finance, China South 1 Finance, China North 2 Ali Gov 1

All DataWorks users

Reverse modeling: Create models from physical tables

Approval center supports managing Data Integration

To protect your data transfer security, the approval center provides approval triggers based on source-destination combinations when saving or running Data Integration tasks, enabling more flexible management of data synchronization processes.

2022.6.15

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong), Singapore, Indonesia (Jakarta), Malaysia (Kuala Lumpur), US (Silicon Valley), US (Virginia), Germany (Frankfurt).

All DataWorks users

Request processing policies for Data Integration tasks

Data Security Guard adds sensitive data lineage visualization

The sensitive data lineage visualization feature includes:

  • Based on data production information, lineage relationships between sensitive columns are parsed, and identification results are propagated among lineage relationships with the same sensitive column type, significantly improving identification efficiency.

  • Lineage relationships can be automatically drawn as sensitive data lineage visualization diagrams to clearly understand data origins and destinations.

Note

This feature is available only for DataWorks Enterprise Edition.

2022.6.14

China (Hangzhou), China (Shanghai)

All DataWorks users

Data lineage (public preview)

Data Security Guard adds anomalous lineage relationship analysis

The anomalous lineage relationship analysis feature includes:

  • The system automatically analyzes anomalous associations between columns based on sensitive column lineage relationships, preventing circumvention of sensitive data identification and usage auditing through character concatenation and splitting.

  • Helps you identify columns that have lineage relationships with query columns but have inconsistent sensitive column type identification results.

2022.6.14

China (Hangzhou), China (Shanghai)

All DataWorks users

Data lineage (public preview)

May

Feature

Description

Release date

Region

Target user

Documentation

Data Security Guard risk identification redesigned (migration required for existing users)

The new risk identification feature provides built-in risk identification scenarios, supports multi-dimensional risk identification combining data classification, operation methods, and personnel permissions, adds alert event aggregation to reduce false positives, supports high, medium, and low granular risk management, and provides comprehensive insight into various enterprise data risks.

Note
  • Risk identification management requires DataWorks Professional Edition or higher.

  • The legacy risk identification management feature is retained until June 30, 2022. After that date, created risk identification rules and related risk data are automatically cleared, and only the new risk identification management feature is available. Export and back up any rules and risk data you need in time.

  • Existing users must migrate to the new version to use the new features.

2022.5.16

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong)

All DataWorks users.

Risk identification management

April

Feature

Description

Release date

Region

Target user

Documentation

DataStudio improves file management capabilities for data development

  • You can directly click Create Node, and the system recommends recently used node types, eliminating the need to manually search for the desired node type.

  • My Favorites is added. Frequently used nodes can be favorited for quick editing or collaborative editing with others.

  • The left-side directory tree is optimized. Uncommitted tasks directly display Commit and Deploy icon buttons for quick task deployment.

2022.4.7

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong)

All DataWorks users.

DataStudio feature guide

Data Quality adds global quality rule batch management

A quality rule list is added with the following features:

  • Displays global Data Quality rules configured in the current workspace, supporting batch enable/disable, subscription, schedule association, and strong/weak rule settings.

  • Combined with the existing Configure by Template rule capabilities, quality rules can be batch created and managed to govern various enterprise data quality issues.

2022.4.11

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong)

All DataWorks users.

Monitoring rule configuration workflow

DataWorks intelligent Data Modeling commercialized

The product starts commercial billing on April 26, 2022. Models created during the preview are retained at no cost, but you cannot perform create, delete, or update operations. To perform these operations, you must purchase DataWorks Standard, Professional, or Enterprise Edition along with the intelligent Data Modeling product.

2022.4.26

All regions

All DataWorks users.

Intelligent data modeling billing

Operation Center intelligent baseline supports more flexible alert management settings

After the intelligent baseline upgrade, the features include:

  • Unified management of baseline instances, events, and baselines is supported.

  • Individual SMS, email, and phone alert rules can be set for each baseline, and duty rosters can be directly associated to avoid complex operations caused by frequent changes of owners.

2022.4.26

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), China (Hong Kong)

All DataWorks users.

Overview

March

Feature

Description

Release date

Region

Target user

Documentation

Deployment Center supports cross-workspace deployment with enhanced deployment management

Deployment Center supports deploying tasks, resources, and functions from one workspace to another, solving the task deployment problem across multiple workspaces.

2022.3.2

All regions

Users in the financial and government sectors with strong deployment management requirements.

Cross-workspace deployment

Data Analysis operation records integrated into security auditing

In DataWorks Data Analysis, the following operation records are integrated into security auditing:

  • Running MaxCompute SQL statements.

  • Downloading SQL execution results.

  • Downloading spreadsheets.

2022.3.20

All regions

All DataWorks users.

Data governance center governance leaderboard optimization

Data governance center governance leaderboard optimization adds the following features:

  • Supports filtering governance items by role.

  • Supports sorting Issues and Check Event by different dimensions.

  • Supports displaying the governance leaderboard for all members in the workspace.

  • Viewing governance issue list details.

  • Viewing resource consumption historical trends and MaxCompute consumption details.

2022.3.21

All regions

Data governance center users participating in the preview.

View governance results

Data Integration task configuration performance optimization

When using exclusive Data Integration resource groups to run One-Click Real-Time Sync to Maxcompute and One-Click Real-Time Sync to Hologres tasks, each task can select more than 1,000 tables for more efficient synchronization.

2022.3.25

All regions

SaaS platforms, financial sector, and other users needing to synchronize large batches of table data.

Prepare a PostgreSQL environment

Intelligent Data Modeling code mode enhancement

  • Dimensional modeling supports adjusting column order of saved models through FML in code mode.

  • Supports automatically generating models from MaxCompute select statements for efficient and quick custom model modifications.

2022.3.27

All regions

All DataWorks users.

Modeling in Script Mode

2021

December

Feature

Description

Release date

Region

Documentation

Data Quality supports batch rule configuration by rule templates

DataWorks Data Quality supports selecting rule templates for batch data quality rule configuration, simplifying configuration:

  • Supports selecting table-level rule templates to batch configure rules for multiple tables.

  • Supports selecting column-level rule templates to batch configure rules for multiple columns.

2021.12.14

All regions.

Configure rules by template

Data governance center adds resource usage analysis feature

DataWorks data governance center adds the resource usage analysis feature, displaying resource consumption overview, anomalies, and details from four dimensions: MaxCompute Storage Resource Consumption, MaxCompute Computing Resource Consumption, DataWorks Scheduling Resource Consumption, and DataWorks Batch Synchronization Consumption.

2021.12.9

All regions.

Data pivoting from the resource type perspective

November

Feature

Description

Release date

Region

Documentation

Data Studio adds resource group orchestration feature

The DataWorks resource group orchestration feature helps you batch modify the scheduling resource groups used by target nodes under a specified workflow during data development. When your workspace has multiple scheduling resource groups, you can use this feature to quickly reassign resource groups to target nodes based on actual business needs, promoting rational resource usage.

2021.11.30

All regions.

Resource Group Orchestration

Data Studio adds batch operations feature

DataWorks supports batch editing operations such as changing owners for nodes, resources, and functions, and supports batch submission and deployment to apply changes to the production environment.

2021.11.11

All regions.

Batch operations

October

Feature

Description

Release date

Region

Documentation

DataWorks intelligent Data Modeling adds reverse modeling and naming dictionary features

  • The naming dictionary provides management of business terms, physical table and column word roots and morphemes, and standardized translation capabilities.

  • The reverse modeling feature helps you reverse engineer models built in other tools into DataWorks dimensional modeling.

2021.10.30

This feature is currently in public preview. Regions with public preview enabled: China (Beijing), China (Shanghai), China (Hangzhou), China (Shenzhen), China (Zhangjiakou), China (Chengdu), Singapore, US (Silicon Valley), Germany (Frankfurt), China (Hong Kong), China East 2 Finance, China South 1 Finance.

Data Studio adds code search feature

The DataWorks code search feature searches for code snippets in nodes by keyword and displays all nodes containing the code snippet along with detailed content. When target table data changes and you need to find the operation source (the task that caused the change), you can use this feature.

2021.10.27

All regions.

Code search

September

Feature

Description

Release date

Region

Documentation

Data Service API assets fully integrated into Data Map

Data Service API assets (wizard, script, registered, and other API types) are integrated into Data Map, supporting enterprise-level and global data interface search and management, including global API search, popular API statistics, independent API detail pages, and API distribution under each data source.

2021.09.30

All regions.

Data governance center released

The DataWorks data governance center automatically discovers and quantifies governance issues across data storage, task computing, code development, data quality, and data security dimensions from global, workspace, and personal perspectives. Using a health score model with governance reports and leaderboards, it helps you effectively resolve governance issues and achieve governance goals.

2021.09.12

This feature is currently in public preview. Regions with public preview enabled: China (Shanghai), China (Hangzhou), China (Beijing), China (Shenzhen).

Overview

August

Feature

Description

Release date

Region

Documentation

Exclusive resource group for Data Service launched in China (Hangzhou) and China (Shanghai)

When API calls in Data Service require high QPS and SLA guarantees, exclusive Data Service resources are needed to ensure call success. The exclusive resource group for Data Service meets high-concurrency, high-frequency API call requirements and returns results in a timely manner.

2021.08.06

China (Hangzhou) and China (Shanghai).

Exclusive resource groups for DataService Studio

DataWorks Migration Assistant officially commercialized on August 1

Migration Assistant helps you quickly replicate development results across different DataWorks versions, primary accounts, regions, and workspaces. It supports migrating scheduled tasks, manual tasks, resources, functions, data sources, table metadata, ad hoc queries, and components. You can choose full export, incremental export, or selective export methods based on business needs.

2021.08.01

All regions.

Migration Assistant

July

Feature

Description

Release date

Region

Documentation

DataWorks data governance module adds approval center with customizable approval workflows

DataWorks approval center is a module for managing data authorization and sensitive behavior control workflows, including approval scope definition and approval process definition, meeting approval requirements of different enterprises under various internal compliance scenarios.

2021.07.16

All regions.

Overview

DataWorks supports dispatching tasks to EMR gateway nodes

Through task advanced parameters, tasks can be dispatched to EMR gateway nodes with EMR load balancing support. Workspace-level task dispatch node settings will be available in the future.

2021.07

All regions.

Create an EMR Hive node

June

Feature

Description

Release date

Region

Documentation

DataWorks supports EMR real-time task development and operations

DataWorks supports developing EMR Spark Streaming and EMR Streaming SQL tasks and performing operations.

Key features include: real-time task development, task dry runs, task submission to the production environment, task error retry, viewing task status and details, starting, terminating, or undeploying tasks, and monitoring failed tasks with alerts.

2021.06

All regions.

Create an EMR Spark Streaming node

One-click migration of EMR data development tasks to DataWorks

DataWorks provides two migration methods to migrate workflows (nodes and schedule configurations), manual tasks, resources, and data sources from EMR clusters to DataWorks workspaces through DataWorks Migration Assistant. You can view migration progress, results, and reports through the DataWorks console Migration Assistant.

2021.06

All regions.

Migrate EMR projects to DataWorks

Operation Center adds resource operations with exclusive resource group watermark monitoring

The resource operations feature monitors the usage of resource groups used for task execution in DataWorks.

2021.06.09

All regions.

Resource O&M

Data Service supports creating APIs from MaxCompute data sources

DataWorks Data Service now supports directly accessing and using MaxCompute tables to create APIs in gray release. API call requests leverage MaxCompute accelerated query (MCQA) for data queries, achieving fast and efficient call returns. (Note: MaxCompute data sources are only supported with exclusive resource group for Data Service.)

2021.06

All regions.

N/A

DataWorks adds alert contact center

You can add target RAM users or RAM roles as alert contacts on the alert contacts page. After adding, DataWorks sends alerts generated by task execution errors to the corresponding contacts, helping you promptly discover and handle exceptional tasks.

2021.06

All regions.

View and configure alert contacts

May

Feature

Description

Release date

Region

Documentation

DataWorks Data Integration adds real-time synchronization to AnalyticDB for MySQL 3.0.

DataWorks supports real-time synchronization of MySQL, OceanBase, PolarDB, and other database data to AnalyticDB for MySQL 3.0. You can perform full synchronization first, then start real-time incremental synchronization for uninterrupted data synchronization. Additionally, real-time synchronization automatically supports adding corresponding columns to the target table when the source adds columns.

2021.05.25

All regions.

Plan and configure resources

Open Message Service (Open Message) public preview

DataWorks provides open messaging capabilities. You can enable the message subscription feature on the DataWorks Open Platform. This feature is available only for DataWorks Enterprise Edition users for public preview. Enterprise Edition users can directly participate in the preview and try the open message feature for free. You can use open messages to obtain metadata and task change events from DataWorks for deep integration between DataWorks and your own systems.

2021.05.21

China (Beijing), China (Hangzhou), China (Shenzhen), and China (Shanghai).

Overview

DataWorks task scheduling adds yearly and month-end schedules

DataWorks scheduled task scheduling adds yearly scheduling and month-end scheduling, enabling scheduling by year, quarter, and the last day of specified months. DataWorks currently supports the following schedule types: Minute, Hour, Sunday, Week, Month, and Year.

2021.05.19

All regions.

Configure time properties

DataWorks supports ClickHouse engine

DataWorks supports the ClickHouse engine, providing ETL operations and management capabilities including data integration, data development, task scheduling, and task operations for the engine.

  • DataWorks supports associating ClickHouse clusters through EMR instance mode or JDBC connection string mode, and associating ClickHouse data sources through JDBC connection string mode.

  • DataWorks supports reading and writing ClickHouse data through Data Integration.

  • The DataWorks ClickHouse SQL node implements a distributed SQL query engine for processing structured data, improving job execution efficiency.

2021.05.15

All regions.

April

Feature

Description

Release date

Region

Documentation

DataWorks Data Integration supports real-time synchronization to AnalyticDB for MySQL 3.0

Real-time synchronization to AnalyticDB for MySQL 3.0 supports synchronizing real-time information of multiple tables in a single task. Real-time synchronization automatically supports DDL synchronization for column additions, where columns added at the source are also added to the target tables.

2021.4.20

All regions.

Create a real-time synchronization solution to synchronize data to AnalyticDB for MySQL 3.0

Data Studio adds FTP Check node

The FTP Check node periodically checks whether a specified file exists via the FTP protocol. If the file exists, it triggers downstream scheduled tasks. If not, it repeats the check at configured intervals until the check stop conditions are met. This node is typically used to pass signals between the DataWorks scheduling system and other scheduling systems.

2021.4.15

China (Beijing), China (Shanghai), China (Hangzhou), China (Shenzhen), China (Zhangjiakou), China (Chengdu), Singapore.

FTP Check node

March

Feature

Description

Release date

Region

Documentation

DataWorks Enterprise Edition adds custom role feature

The custom role feature enables you to define permission capabilities tailored to actual business needs.

2021.3.22

All regions.

Manage permissions on workspace-level services

DataWorks Data Integration supports Kerberos authentication file mechanism

For data sources such as Hive and Kafka that sometimes require Kerberos authentication files for access, Data Integration supports the Kerberos authentication file mechanism, allowing you to freely upload data source authentication files and use them during configuration to ensure data source access security.

2021.3.16

All regions.

Appendix: Configure Kerberos authentication

DataWorks console adds new Security Center feature

DataWorks Security Center helps you quickly build security capabilities for data content, personal privacy, and related areas on the platform, meeting various security requirements for high-risk scenarios (such as auditing) without additional configuration.

2021.03.13

All regions.

Overview

Operation Center adds DAG aggregation view and upstream/downstream analysis feature

You can aggregate nodes in the DAG by dimensions such as workspace, owner, and priority to view the total number of nodes in the target dimension. You can also perform upstream/downstream analysis on specific nodes to quickly locate upstream nodes blocking the current node and view the number of downstream nodes, giving you insight into task execution status.

2021.03.10

China (Shenzhen).

Manage scheduled tasks

February

Feature

Description

Release date

Region

Documentation

Data Modeling commercialized

DataWorks Data Modeling Datablau is officially commercialized. You can purchase this module to integrate data model design management, standard implementation, and other capabilities into the DataWorks standardized development workflow, helping you achieve data asset value output, build a solid data foundation throughout the data lifecycle, and provide strong support for data value realization.

2021.02.20

All regions.

DATABLAU data modeling billing standards

Data discovery supports batch creation of metadata collectors

Data Map > Collect Metadata supports batch creation of metadata collectors to help you quickly view table internal structures and related relationships in Data Map.

2021.02.17

All regions.

Collect metadata from an EMR data source

Migration Assistant supports Airflow

Migration Assistant supports migrating tasks from the Airflow scheduling system to DataWorks.

2021.02.16

All regions.

Export tasks from open-source engines

Data Service adds metering API

The Data Service metering API, including the metering dashboard and metering details, provides various visual charts and statistics, including the total number of APIs and total call count under a workspace, helping you understand API call status from a global perspective. You can also view monitoring charts for individual APIs with detailed information such as API Gateway status codes and Data Service error codes.

2021.02.16

China (Beijing).

DataWorks console adds Open Platform feature

The DataWorks console Open Platform feature displays Open API metering reports for viewing Open API call details for specified dates.

2021.02.13

All regions.

DataWorks Open Platform Open API report

January

Feature

Description

Release date

Region

Documentation

DataWorks Data Integration adds support for HTTP data sources

Some data sources only provide data through HTTP REST APIs or receive data. For such data sources, Data Integration supports batch reading and writing.

2021.1.4

All regions.

RestAPI Reader

2020

December

Feature

Description

Release date

Region

Documentation

DataWorks Data Integration adds full and incremental synchronization to Elasticsearch

Supports one-time full synchronization of all or some tables from a specified database to Elasticsearch, with subsequent real-time incremental synchronization mode to continuously synchronize new data to Elasticsearch.

2020.12.30

All regions.

Synchronize an entire MySQL database to Elasticsearch

September

Feature

Description

Release date

Region

Documentation

DataWorks Data Integration adds real-time synchronization feature

The real-time data synchronization feature synchronizes data changes from part or all of the source database tables to the target database in real time, keeping the target database in sync with the source database. Solutions enable full and incremental integrated synchronization.

2021.4.15

All regions.

July

Feature

Description

Release date

Region

Documentation

Exclusive resource group monitoring feature

DataWorks exclusive resource groups support resource watermark monitoring, providing detailed resource usage trends. You can view task execution and queuing status to support fine-grained resource group scaling.

2020.07.17

All regions.

View and set alert resources

Open API public preview for customizing DataWorks features

DataWorks opens tenant, metadata, data development, Operation Center, Data Quality, Data Service, and other APIs, giving you more freedom to use DataWorks.

Note

You must purchase DataWorks Enterprise Edition or higher to use the Open API feature.

2020.07.16

China (Hangzhou), China (Shanghai), China (Shenzhen), China (Beijing), and China (Zhangjiakou).

Overview

Migration Assistant public preview

Migration Assistant supports migrating scheduled tasks, manual tasks, resources, functions, data sources, table metadata, ad hoc queries, and components. You can choose full export, incremental export, or selective export methods based on business needs.

2020.07

China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Chengdu), and Singapore.

Migration Assistant

Data Service update

DataWorks Data Service releases a brand-new directory tree structure.

2020.07.28

  • The function and filter features are only available in the China (Shanghai) region and require DataWorks Professional Edition or higher.

  • The API orchestration feature is only available in the China (Shanghai) region and requires DataWorks Enterprise Edition or higher.

DataService Studio

June

Feature

Description

Release date

Region

Documentation

Query from data source feature

You can use the query from data source feature on the spreadsheet editing page to quickly read data from data sources for analysis.

2020.06.09

China (Shanghai).

Analyze data

April

Feature

Description

Release date

Region

Documentation

Operation Center supports phone alert feature

Operation Center supports three alert methods: Text Message, Email, and Telephone.

Important

You must purchase DataWorks Professional Edition or higher to use the phone alert feature.

2020.04.15

All regions.

Rule management