Getting started with DataService Studio

更新时间:
复制 MD 格式

DataService Studio in DataWorks is a flexible, lightweight, secure, and stable platform for building data APIs. Acting as a bridge between databases and data applications, DataService Studio provides comprehensive data service and sharing capabilities for individuals, teams, and enterprises. It helps you centrally manage both internal and external API services. For example, you can quickly generate an API from a data table query or register an existing API on the platform for unified publishing and management.

Features

As a bridge between databases and data applications, DataService Studio offers comprehensive data service and sharing capabilities. The service uses a Serverless architecture, allowing you to focus on the query logic of your APIs without managing underlying infrastructure like runtime environments. DataService Studio provisions computing resources for you and supports elastic scaling, ensuring zero operational overhead. It is tightly integrated with API Gateway, allowing you to publish APIs to API Gateway with a single click. Together, DataService Studio and API Gateway deliver a secure, stable, cost-effective, and easy-to-use service for open data sharing.

Note
  • If you encounter issues while using DataService Studio, click the group invitation to join the DataWorks DingTalk group for assistance.

  • DataService Studio is only supported on PCs with Chrome version 69 or later.

image

Key concepts

  • API (Application Programming Interface): An API is an interface that allows applications, software, and systems to interact with a data source. DataService Studio APIs support read operations, enabling continuous data queries from databases and data tables.

  • Function: A function acts as an API filter to process the request or response parameters of an API. When using a function as an API filter, the function type for both the pre-filter and post-filter must be the same. Using different function types for the pre- and post-filters of the same API is not supported.

  • Data Push: Data Push is a feature in DataWorks that allows you to write SQL to query data sources such as MySQL, PostgreSQL, Hologres, MaxCompute, and ClickHouse. You can then combine the query results with rich text or tables and automatically push the content on a regular schedule to channels like DingTalk groups, Lark groups, WeCom, email, or Microsoft Teams.

  • Resource groups for DataService Studio: For API calls that require high queries per second (QPS) and service level agreement (SLA) guarantees, you must use exclusive data service resources. An exclusive resource group for DataService Studio can handle high-concurrency, high-frequency API calls and return data in a timely manner.

Billing

DataService Studio charges for API calls by volume through the following billing models:

  • Serverless resource group:

    • Pay-as-you-go: When using a pay-as-you-go Serverless resource group, you must configure the Currently Occupied CUs for the Occupied CUs usage. For more information, see Allocate CU quotas to tasks. Billing starts as soon as you allocate the CUs and is calculated as Number of Currently Occupied CUs × Duration of Allocation.

    • Subscription: When using a subscription Serverless resource group, you must configure Guaranteed CUs for the Minimum CUs usage. For more information, see Allocate CU quotas to tasks. You are billed for the subscribed number of CUs.

  • Exclusive resource group for DataService Studio: You are charged based on the specifications of the resource group. You can evaluate the expected API call volume for your business and select an appropriate resource group size. An exclusive resource group for DataService Studio is dedicated to a single enterprise. For more information, see Billing of exclusive resource groups for DataService Studio.

Capability overview

API development

Step

Name

Description

1

Create a data source and configure network connectivity

Before you create an API in DataService Studio, you must add your database or data warehouse as a DataWorks data source. When you develop the API, DataService Studio accesses this data source to retrieve the table schema, which helps you configure request and response parameters. When the API is called, DataService Studio accesses the data source to execute the query. For a list of supported data sources, see List of data sources for DataService Studio.

Important

If the connectivity test fails, check your configuration by following the instructions in Network connections and whitelists.

2

Create a business process

DataService Studio uses a business process to organize API development around specific business units. It also uses the business process to manage files such as APIs and functions. You must create a business process before you can create an API.

3

Create an API

  • Quickly generate a data API from a data table

    • DataService Studio provides a Wizard Mode with a visual interface for generating data APIs from tables in various data sources. You can quickly configure an API without writing any code. For more information, see Generate an API from a data source (API Gateway).

    • To meet the custom query needs of advanced users, DataService Studio offers a Code Editor mode that allows you to write your own SQL queries for the API. This mode supports advanced features such as table joins, complex queries, and aggregate functions. For more information, see Generate an API in Code editor mode.

  • Register an existing API: Quickly register an existing API on the platform for unified management and publishing. For more information, see Register an external API service.

4

Create a function

This optional step allows you to use Function Compute to process API request and response parameters.

5

Test the API

You must test an API before it can be published.

6

Test, publish, and manage API versions

After an API passes testing, you can publish it to API Gateway, which allows other services to call it. After an API is published, you can also list it for sale on the Alibaba Cloud API Marketplace with a single click, helping you quickly monetize your data and create a closed commercial loop.

Note
  • API Gateway is a managed service that covers the entire lifecycle of an API, including publishing, management, O&M, and sales. It helps you aggregate microservices, separate front-end and back-end systems, and integrate systems in a simple, fast, cost-effective, and low-risk way to open up functions and data to partners and developers. For more information, see Product introduction.

    API Gateway acts as the final line of defense for APIs that are exposed to external systems or called within your own applications. It provides services such as permission management, traffic shaping, access control, and metering. Typically, APIs generated and registered in DataService Studio must be published to API Gateway before they can be served externally. DataService Studio is integrated with API Gateway, supporting one-click publishing.

  • The Alibaba Cloud API Marketplace covers seven major categories, including finance, artificial intelligence, e-commerce, transportation and geography, lifestyle services, business management, and public affairs. Thousands of API products are currently available for sale, providing a platform to quickly monetize your data. For more information, see Alibaba Cloud API Marketplace.

7

Call an API

An application (app) represents your identity when calling an API. To call an API, the app must be authorized. You can be granted authorization for an API by others, or you can authorize others to use your API to share data. For more information, see Authorize an API. You can also use API Gateway to call APIs exposed by other Alibaba Cloud users or third-party service providers. API Gateway provides a range of management and support services.

Data Push

DataService Studio supports the Data Push feature, which allows you to proactively send data to a target audience based on a schedule. You can create a Data Push task to customize and orchestrate content, such as rich text and tables, from various data sources. You can also set the push frequency based on your business needs and configure the recipients and channels for different consumers. This helps improve data delivery efficiency and expands data consumption scenarios. For more information, see Data Push.

Quick start

Prerequisites

Step 1: Create data source and configure network

Before you create an API in DataService Studio, you must add your database or data warehouse as a DataWorks data source and ensure that the DataService Studio resource group can connect to your destination data source. This allows DataWorks to access the data source when an API is called.

When you develop an API, DataService Studio accesses this data source to retrieve the table schema, which helps you configure request and response parameters. When the API is called, DataService Studio accesses the data source to execute the query. For a list of supported data sources, see List of data sources for DataService Studio.

This example shows how to create a MySQL data source named rds_workshop_log and test its connectivity with a Serverless resource group.

  1. Go to the DataWorks console, select the target workspace, and click Go to Management Center.

  2. In the left-side navigation pane, click Data Sources.

  3. Create a data source and test its network connectivity.

    1. Click Add Connection and select a data source type.

      This example uses MySQL.

    2. Configure the data source information.

      Configure the parameters as prompted on the page. For more information, see Configure a MySQL data source.

    3. Select a Serverless resource group and test the connectivity between the resource group and the data source.

      Note
    4. Click Complete to create the data source.

Step 2: Create an API group

DataService Studio provides services to external systems through API Gateway. In API Gateway, a group is a management unit for APIs and is associated with the domain name of the call address. Therefore, you must first create an API group in API Gateway before you create an API in DataService Studio.

  1. Go to the API Gateway console. On the Group Management page, click Create Group.

  2. Enter the group information and complete the creation. This example creates a group named Workshop.

Step 3: Create a business process

In DataService Studio, business processes are used to develop APIs based on business units and to organize and manage files, such as APIs and functions. Therefore, you must first create a business process before you create an API.

  1. Log on to the DataWorks console. In the target region, click Data Analysis and Service > DataService Studio in the left-side navigation pane. Select a workspace from the drop-down list and click Go to DataService Studio.

  2. Create a business process.

    On the Data Services > Service Development page, create a business process in one of the following ways:

    • Method 1: Click the 新建 icon and select Create Workflow.

    • Method 2: In the left-side navigation pane, right-click the Workflow folder and select Create Workflow.

  3. Configure the parameters for the business process.

    Configuration

    Description

    Workflow Name

    Enter a custom name based on your business scenario. This example creates a business process named DataService.

    API grouping

    Select an API group to manage your APIs. This example uses the Workshop group created in Step 2.

    Note

    An API group is a collection of APIs for a specific feature or scenario and the smallest unit for API management in API Gateway. If you need to create a new group, see Step 2: Create an API group.

  4. Click Determine to create the business process.

    After the business process is created, you can view its details in the folder in the navigation pane on the left of the DataService Studio page.

Step 4: Generate an API

After you create the business process, you can create the corresponding business APIs within it.

  1. Generate an API.

    On the Data Services > Service Development page, generate an API in one of the following ways:

    • Method 1: Click the 新建 icon and choose Create API > Generate API.

    • Method 2: In the left-side navigation pane, right-click the destination business process and choose Create > API > Generate API.

  2. Define the API properties.

    In the Generate API dialog box, configure the API information. The following table describes the main parameters.

    Configuration Item

    Description

    Location

    The directory within the business process where the API is stored. Click to select a folder.

    Important

    We recommend selecting the destination folder first. If you cannot select one, create a business process first.

    API Mode

    Select the mode to create the API. You can choose the codeless UI or the code editor.

    • Codeless UI: Quickly configure a data API without writing code.

    • Code editor: Supports advanced features such as table joins, complex queries, and aggregate functions. You can write custom SQL queries for the API to meet advanced query requirements.

    This topic uses the codeless UI as an example. For more information about the two modes, see Overview of API generation.

    API Name

    Enter a custom name for the API. This topic uses rpt_user_info_all_d as an example.

    APIPath

    The storage path of the API. This is the URI of the request relative to the service host. This path is used to construct the API call address. This topic uses /user as an example.

    Protocol

    HTTP and HTTPS are supported.

    To call the API over HTTPS, bind an independent domain name and upload an SSL Certificate in the API Gateway console after you publish the API to the gateway. For more information, see Support for HTTPS.

    Request Method

    GET and POST are supported.

    Note
    • If you select GET as the request method, you can only set the Request Parameters of a Parameter Position to QUERY.

    • If you select POST as the request method, you can set the Request Parameters of a Parameter Position to QUERY or BODY.

    Response Type

    Only JSON is supported.

    Visible Scope

    Defines the visibility of the API after it is created. Options are Work space and Private.

    • Work space: The API is visible to all members in the current workspace.

    • Private: The API is visible only to the API owner. Authorization is not supported.

    This example sets the visible scope to Work space.

  3. Configure the API parameters.

    On the API editing page, configure the API parameters. This example uses the following configuration.

    1. Select the data source for the API.

      In the Table area, select the data source table for generating the API.

      Parameter

      Description

      Data Source Type

      Select the type of data source from which to create the API. This topic uses MySQL as an example.

      Data Source Name

      Select the specific data source of the chosen type to generate the API from. This example uses the rds_workshop_log data source created in Step 1: Create a data source and configure network connectivity.

      Data Source Environment

      Workspaces in standard mode allow access to data sources in either the development or production environment. For more information about the differences between the two environments, see Differences between workspace modes. This example selects the Production Environment.

      Data Table Name

      Select the data table in the data source from which to generate the API. This topic uses the rpt_user_info_all_d table in the rds_workshop_log data source as an example.

    2. Define the request and response parameters.

      After you select a table, its fields appear in the Select Parameters area. You can then choose the fields to use as request and response parameters. To sort the fields, add the target fields to the Sort field area and adjust their order.

      In this example, the request parameter is uid, and the response parameters are all fields in the table.

      Note

      In the right-side navigation pane, click Request Parameters or Response Parameters to modify parameter settings, such as field type, sample value, and default value. Configure these settings based on your actual needs.

  4. Configure the API resource group.

    On the API editing page, click Resource Group for DataService Studio on the right to configure the resource group. Because the default public resource group for DataService Studio is being discontinued, new APIs must use an exclusive service resource group. Select either a Serverless resource group (recommended) with a configured quota for DataService Studio, or an exclusive resource group for DataService Studio.

Step 5: Test the API

You must test an API before it can be published. After the API is generated, click Test in the upper-right corner of the API editing page to verify that the request and response are as expected. This example sets the value of the uid request parameter to 0016359810821 to test the result. For more information about testing, see Test an API.

Step 6: Publish the API

An API that passes testing must be published before other services can call it. In the upper-right corner of the API editing page, click Submission and select the version to publish. After the API is published, you can view it in API Gateway. For more information about publishing, see Test, publish, and manage API versions.

Step 7: Call the API

The following authentication methods are supported for calling APIs. You can select one based on your requirements:

  • Simple Authentication: Offers low security. It is suitable for scenarios such as reports and data dashboards.

  • Encrypted Signature Authentication: Offers high security. It is suitable for application development scenarios.

  1. In the top menu bar of DataService Studio, click Service Management to go to the service management page.

  2. In the left-side navigation pane, select Call APIs.

    Published APIs are available in API Gateway. You can call the API using an API Gateway SDK. For more information, see Examples of calling an API from a client.

Step 8: View and manage APIs

On the Service Management page, you can view published APIs and perform management operations such as unpublishing, authorization, and viewing call details.

Area

Description

API Management

On the Service Management > Manage APIs page, you can view currently published APIs, APIs you are authorized to use, and APIs you have authorized to others. You can also perform operations such as Unpublish, Authorization, and Test on APIs for which you have permissions.

API Metering

On the Service Management > API Statistics page, you can perform the following operations:

  • Statistics Dashboard: View a summary of API call information, including the total number of APIs, status code distribution, error code distribution, overall metering, and resource group allocation.

  • Statistics Details: Provides various charts and statistics. You can view monitoring charts for a single API, including API Gateway status codes, DataService Studio error codes, app request counts, traffic bandwidth, and average response time.

  • Log Analysis: Use the log analysis feature to analyze call requests. If a call request is abnormal, intelligent analysis can help you quickly locate the problem and provide a diagnosis and suggestions.