DataService Studio in DataWorks is a flexible, lightweight, secure, and stable platform for building data APIs. Acting as a bridge between databases and data applications, DataService Studio provides comprehensive data service and sharing capabilities for individuals, teams, and enterprises. It helps you centrally manage both internal and external API services. For example, you can quickly generate an API from a data table query or register an existing API on the platform for unified publishing and management.
Features
As a bridge between databases and data applications, DataService Studio offers comprehensive data service and sharing capabilities. The service uses a Serverless architecture, allowing you to focus on the query logic of your APIs without managing underlying infrastructure like runtime environments. DataService Studio provisions computing resources for you and supports elastic scaling, ensuring zero operational overhead. It is tightly integrated with API Gateway, allowing you to publish APIs to API Gateway with a single click. Together, DataService Studio and API Gateway deliver a secure, stable, cost-effective, and easy-to-use service for open data sharing.
-
If you encounter issues while using DataService Studio, click the group invitation to join the DataWorks DingTalk group for assistance.
-
DataService Studio is only supported on PCs with Chrome version 69 or later.
Key concepts
-
API (Application Programming Interface): An API is an interface that allows applications, software, and systems to interact with a data source. DataService Studio APIs support read operations, enabling continuous data queries from databases and data tables.
-
Function: A function acts as an API filter to process the request or response parameters of an API. When using a function as an API filter, the function type for both the pre-filter and post-filter must be the same. Using different function types for the pre- and post-filters of the same API is not supported.
-
Data Push: Data Push is a feature in DataWorks that allows you to write SQL to query data sources such as MySQL, PostgreSQL, Hologres, MaxCompute, and ClickHouse. You can then combine the query results with rich text or tables and automatically push the content on a regular schedule to channels like DingTalk groups, Lark groups, WeCom, email, or Microsoft Teams.
-
Resource groups for DataService Studio: For API calls that require high queries per second (QPS) and service level agreement (SLA) guarantees, you must use exclusive data service resources. An exclusive resource group for DataService Studio can handle high-concurrency, high-frequency API calls and return data in a timely manner.
Billing
DataService Studio charges for API calls by volume through the following billing models:
-
Serverless resource group:
-
Pay-as-you-go: When using a pay-as-you-go Serverless resource group, you must configure the Currently Occupied CUs for the Occupied CUs usage. For more information, see Allocate CU quotas to tasks. Billing starts as soon as you allocate the CUs and is calculated as
Number of Currently Occupied CUs × Duration of Allocation. -
Subscription: When using a subscription Serverless resource group, you must configure Guaranteed CUs for the Minimum CUs usage. For more information, see Allocate CU quotas to tasks. You are billed for the subscribed number of CUs.
-
-
Exclusive resource group for DataService Studio: You are charged based on the specifications of the resource group. You can evaluate the expected API call volume for your business and select an appropriate resource group size. An exclusive resource group for DataService Studio is dedicated to a single enterprise. For more information, see Billing of exclusive resource groups for DataService Studio.
The public resource group for DataService Studio is no longer recommended. For more information, see Discontinuation of Public Resource Groups for DataService Studio in DataWorks.
Capability overview
API development
|
Step |
Name |
Description |
|
1 |
Before you create an API in DataService Studio, you must add your database or data warehouse as a DataWorks data source. When you develop the API, DataService Studio accesses this data source to retrieve the table schema, which helps you configure request and response parameters. When the API is called, DataService Studio accesses the data source to execute the query. For a list of supported data sources, see List of data sources for DataService Studio. Important
If the connectivity test fails, check your configuration by following the instructions in Network connections and whitelists. |
|
|
2 |
DataService Studio uses a business process to organize API development around specific business units. It also uses the business process to manage files such as APIs and functions. You must create a business process before you can create an API. |
|
|
3 |
|
|
|
4 |
This optional step allows you to use Function Compute to process API request and response parameters. |
|
|
5 |
You must test an API before it can be published. |
|
|
6 |
After an API passes testing, you can publish it to API Gateway, which allows other services to call it. After an API is published, you can also list it for sale on the Alibaba Cloud API Marketplace with a single click, helping you quickly monetize your data and create a closed commercial loop. Note
|
|
|
7 |
An application (app) represents your identity when calling an API. To call an API, the app must be authorized. You can be granted authorization for an API by others, or you can authorize others to use your API to share data. For more information, see Authorize an API. You can also use API Gateway to call APIs exposed by other Alibaba Cloud users or third-party service providers. API Gateway provides a range of management and support services. |
Data Push
DataService Studio supports the Data Push feature, which allows you to proactively send data to a target audience based on a schedule. You can create a Data Push task to customize and orchestrate content, such as rich text and tables, from various data sources. You can also set the push frequency based on your business needs and configure the recipients and channels for different consumers. This helps improve data delivery efficiency and expands data consumption scenarios. For more information, see Data Push.
Quick start
Prerequisites
-
API Gateway must be activated. For more information, see Activate API Gateway.
-
A destination data source is created. This topic uses a MySQL data source as an example. The data source provides the input data required to generate the API. For more information, see Configure a data source.
-
A Serverless resource group is created and a quota is configured for DataService Studio. For more information, see Configure a quota for DataService Studio in a Serverless resource group.
Step 1: Create data source and configure network
Before you create an API in DataService Studio, you must add your database or data warehouse as a DataWorks data source and ensure that the DataService Studio resource group can connect to your destination data source. This allows DataWorks to access the data source when an API is called.
When you develop an API, DataService Studio accesses this data source to retrieve the table schema, which helps you configure request and response parameters. When the API is called, DataService Studio accesses the data source to execute the query. For a list of supported data sources, see List of data sources for DataService Studio.
This example shows how to create a MySQL data source named rds_workshop_log and test its connectivity with a Serverless resource group.
-
Go to the DataWorks console, select the target workspace, and click Go to Management Center.
-
In the left-side navigation pane, click Data Sources.
-
Create a data source and test its network connectivity.
-
Click Add Connection and select a data source type.
This example uses MySQL.
-
Configure the data source information.
Configure the parameters as prompted on the page. For more information, see Configure a MySQL data source.
-
Select a Serverless resource group and test the connectivity between the resource group and the data source.
Note-
If you have high-concurrency tasks that cannot be run during off-peak hours, you must use dedicated resources to ensure high-concurrency and high-frequency API calls. We recommend that you purchase and use a Serverless resource group from DataWorks.
When you use a Serverless resource group, you must configure a quota for DataService Studio in the Serverless resource group.
-
If the connectivity test fails, check your configuration by following the instructions in Network connections and whitelists.
-
-
Click Complete to create the data source.
-
Step 2: Create an API group
DataService Studio provides services to external systems through API Gateway. In API Gateway, a group is a management unit for APIs and is associated with the domain name of the call address. Therefore, you must first create an API group in API Gateway before you create an API in DataService Studio.
-
Go to the API Gateway console. On the Group Management page, click Create Group.
-
Enter the group information and complete the creation. This example creates a group named
Workshop.
Step 3: Create a business process
In DataService Studio, business processes are used to develop APIs based on business units and to organize and manage files, such as APIs and functions. Therefore, you must first create a business process before you create an API.
-
Log on to the DataWorks console. In the target region, click in the left-side navigation pane. Select a workspace from the drop-down list and click Go to DataService Studio.
-
Create a business process.
On the page, create a business process in one of the following ways:
-
Method 1: Click the
icon and select Create Workflow. -
Method 2: In the left-side navigation pane, right-click the Workflow folder and select Create Workflow.
-
-
Configure the parameters for the business process.
Configuration
Description
Workflow Name
Enter a custom name based on your business scenario. This example creates a business process named DataService.
API grouping
Select an API group to manage your APIs. This example uses the Workshop group created in Step 2.
NoteAn API group is a collection of APIs for a specific feature or scenario and the smallest unit for API management in API Gateway. If you need to create a new group, see Step 2: Create an API group.
-
Click Determine to create the business process.
After the business process is created, you can view its details in the folder in the navigation pane on the left of the DataService Studio page.
Step 4: Generate an API
After you create the business process, you can create the corresponding business APIs within it.
-
Generate an API.
On the page, generate an API in one of the following ways:
-
Method 1: Click the
icon and choose . -
Method 2: In the left-side navigation pane, right-click the destination business process and choose .
-
-
Define the API properties.
In the Generate API dialog box, configure the API information. The following table describes the main parameters.
Configuration Item
Description
Location
The directory within the business process where the API is stored. Click to select a folder.
ImportantWe recommend selecting the destination folder first. If you cannot select one, create a business process first.
API Mode
Select the mode to create the API. You can choose the codeless UI or the code editor.
-
Codeless UI: Quickly configure a data API without writing code.
-
Code editor: Supports advanced features such as table joins, complex queries, and aggregate functions. You can write custom SQL queries for the API to meet advanced query requirements.
This topic uses the codeless UI as an example. For more information about the two modes, see Overview of API generation.
API Name
Enter a custom name for the API. This topic uses
rpt_user_info_all_das an example.APIPath
The storage path of the API. This is the URI of the request relative to the service host. This path is used to construct the API call address. This topic uses
/useras an example.Protocol
HTTP and HTTPS are supported.
To call the API over HTTPS, bind an independent domain name and upload an SSL Certificate in the API Gateway console after you publish the API to the gateway. For more information, see Support for HTTPS.
Request Method
GET and POST are supported.
Note-
If you select GET as the request method, you can only set the Request Parameters of a Parameter Position to QUERY.
-
If you select POST as the request method, you can set the Request Parameters of a Parameter Position to QUERY or BODY.
Response Type
Only JSON is supported.
Visible Scope
Defines the visibility of the API after it is created. Options are Work space and Private.
-
Work space: The API is visible to all members in the current workspace.
-
Private: The API is visible only to the API owner. Authorization is not supported.
This example sets the visible scope to Work space.
-
-
Configure the API parameters.
On the API editing page, configure the API parameters. This example uses the following configuration.
-
Select the data source for the API.
In the Table area, select the data source table for generating the API.
Parameter
Description
Data Source Type
Select the type of data source from which to create the API. This topic uses MySQL as an example.
Data Source Name
Select the specific data source of the chosen type to generate the API from. This example uses the
rds_workshop_logdata source created in Step 1: Create a data source and configure network connectivity.Data Source Environment
Workspaces in standard mode allow access to data sources in either the development or production environment. For more information about the differences between the two environments, see Differences between workspace modes. This example selects the Production Environment.
Data Table Name
Select the data table in the data source from which to generate the API. This topic uses the
rpt_user_info_all_dtable in therds_workshop_logdata source as an example. -
Define the request and response parameters.
After you select a table, its fields appear in the Select Parameters area. You can then choose the fields to use as request and response parameters. To sort the fields, add the target fields to the Sort field area and adjust their order.
In this example, the request parameter is
uid, and the response parameters are all fields in the table.NoteIn the right-side navigation pane, click Request Parameters or Response Parameters to modify parameter settings, such as field type, sample value, and default value. Configure these settings based on your actual needs.
-
-
Configure the API resource group.
On the API editing page, click Resource Group for DataService Studio on the right to configure the resource group. Because the default public resource group for DataService Studio is being discontinued, new APIs must use an exclusive service resource group. Select either a Serverless resource group (recommended) with a configured quota for DataService Studio, or an exclusive resource group for DataService Studio.
Step 5: Test the API
You must test an API before it can be published. After the API is generated, click Test in the upper-right corner of the API editing page to verify that the request and response are as expected. This example sets the value of the uid request parameter to 0016359810821 to test the result. For more information about testing, see Test an API.
Step 6: Publish the API
An API that passes testing must be published before other services can call it. In the upper-right corner of the API editing page, click Submission and select the version to publish. After the API is published, you can view it in API Gateway. For more information about publishing, see Test, publish, and manage API versions.
Step 7: Call the API
The following authentication methods are supported for calling APIs. You can select one based on your requirements:
Simple Authentication: Offers low security. It is suitable for scenarios such as reports and data dashboards.
Encrypted Signature Authentication: Offers high security. It is suitable for application development scenarios.
-
In the top menu bar of DataService Studio, click Service Management to go to the service management page.
-
In the left-side navigation pane, select Call APIs.
Published APIs are available in API Gateway. You can call the API using an API Gateway SDK. For more information, see Examples of calling an API from a client.
Step 8: View and manage APIs
On the Service Management page, you can view published APIs and perform management operations such as unpublishing, authorization, and viewing call details.
Area | Description |
|
On the page, you can view currently published APIs, APIs you are authorized to use, and APIs you have authorized to others. You can also perform operations such as Unpublish, Authorization, and Test on APIs for which you have permissions. |
|
|
On the page, you can perform the following operations:
|