DocsICP FilingConsole
Documentation
Home Page DataWorks User Guide Use cases Best practices Data analysis solutions Connecting DataWorks DataService Studio to DataV

Connecting DataWorks DataService Studio to DataV

Updated at:
Copy as MD
Product
My Favorites

This topic describes how to connect DataV to DataWorks DataService Studio. You can use DataService Studio with a Hologres connection to develop and generate an API, and then call the API in DataV to visualize the analysis results of data from MaxCompute.

Background

MaxCompute is a fast, fully managed, and enterprise-level data warehouse solution developed by Alibaba Group that processes data at terabyte, petabyte, and even exabyte scales. As data collection methods diversify and industry data accumulates, data volumes have grown beyond the capacity of traditional software. MaxCompute stores and computes large volumes of structured data and has reliably supported all of Alibaba Group's offline analysis workloads for years.

Previously, visualizing the analysis results of massive datasets in DataV required building a complex and costly workflow to automatically compute offline data and import it into a MySQL database. Now, DataWorks provides an end-to-end data development platform that includes Data Integration > DataStudio > Data Services. By combining these services with MaxCompute, you can quickly build an enterprise data warehouse.

With DataWorks DataService Studio, you can quickly generate an API from a data table. Using the visual wizard, you can create an API without writing any code. You can then call this API from DataV to display data analysis results on a dashboard, creating an efficient workflow from data warehousing to data visualization.

Prerequisites

Before you begin, ensure that a data source is ready and DataV is activated. For more information, see What is DataV?.

Create a data source

DataService Studio supports a wide range of data source types:

  • Relational databases: ApsaraDB RDS, DRDS, MySQL, PostgreSQL, Oracle, and SQL Server

  • Analytic databases: AnalyticDB

  • NoSQL databases: Tablestore and MongoDB

  1. Log on to the DataWorks console. In the target region, click Data Analysis and Service > DataService Studio in the left-side navigation pane. Select a workspace from the drop-down list and click Go to DataService Studio.

  2. Hover over the image icon and click Add Data Source. You are redirected to the Data Sources page.

  3. Click Add Connection. In the Add Connection dialog box, select Hologres as the data source type.

    This topic uses a Hologres data source as an example. With a Hologres data source, you can query data in MaxCompute in real time.

    1. Configure the parameters in the Add Hologres Data Source dialog box.

      Note

      If your workspace is in standard mode, you must configure both the Development and Production environments.

      Parameter

      Description

      Data Source Name

      The name must consist of letters, digits, and underscores (_), and cannot start with a digit or an underscore.

      Description

      A brief description of the data source, limited to 80 characters.

      Hologres Instance

      The ID of the Hologres instance to connect to. Switch the Region to find the target Hologres instance.

      Note

      If no Hologres instance is available, click Create Instance and follow the prompts.

      Database Name

      The name of the Hologres database.

    2. Click Test Connectivity.

    3. After the connectivity test passes, click Complete.

Create an API

After you create the data source, go to the Data Services page. This topic uses an example of generating an API in Wizard Mode to describe how to create a new API.

  1. Click the icon in the upper-left corner and choose All Products > Data Services to go to the Data Services page.

  2. Hover over the image icon and choose Create API > Generate API.

  3. In the Generate API dialog box, configure the parameters.

    Parameter

    Description

    API Name

    The name must be 4 to 50 characters in length, can contain Chinese characters, letters, digits, and underscores (_), and must start with a Chinese character or a letter.

    API path

    The path where the API is stored, for example, /user.

    Protocol

    HTTP and HTTPS are supported.

    Request Method

    GET and POST are supported.

    Response Type

    Only JSON is supported.

    Description

    A brief description of the API.

    Location

    The folder in the business flow where the API is stored.

  4. When you finish, click Confirm to go to the API parameter configuration page.

Configure API parameters

  1. In the Table section, set Data Source Type to Hologres, and select a Data Source Name and Data Table Name.

    Note

    • You must first configure the data source in Data Integration. You can search for a table by its name in the table name drop-down list.

    • After you create an API, you are automatically redirected to the table configuration page where you can configure the table directly.

  2. After you select a table, the Select Parameters section below automatically lists all fields in that table. Select the desired fields and use the Set as Req Param and Set as Resp Param options to add them to the respective parameter lists.

  3. Configure the request parameters.

    In the right-side pane, click the Request Parameters tab. Set the Parameter Name, Parameter Type, Operator, Required switch, Sample Value, Default Value, and Description.

  4. Configure the response parameters.

    In the right-side pane, click the Response Parameters tab. Set the Alias, Parameter Type, Sample Value, and Description. You can also use advanced settings such as the Filter.

    Note

    • If you use a public resource group, each page can return a maximum of 2,000 records with pagination enabled.

    • If you use an exclusive resource group, each page can return a maximum of 10,000 records with pagination enabled.

Test API

After you configure and save the API parameters, click Test in the top toolbar to start testing the API.

Enter the parameter values and click Start Test to send an online API request. You can view the request details and response in the right-side pane. If the test fails, review the error message, make the necessary changes, and test again.

Publish API

  1. After the API passes the test, return to the Service Development page. Click Submission to generate API Version information.

  2. In the Version information, find an API version with the Status of Can Be Published, and click Publish in the Operation column to generate a data API.

  3. After publishing, click Service Management at the top to view the API details.

If you need to call the API, go to the Service Management > Call APIs page. DataService Studio provides two authentication methods: simple authentication (AppCode) and encrypted signature authentication (AppKey & AppSecret). You can choose either method. The following section describes how to call a DataService Studio API in DataV.

Using DataService Studio APIs in DataV

Add DataService Studio as a data source

  1. Log on to the DataV console.

  2. In the left-side navigation pane, choose Prepare data > Data Sources. On the Data Sources page that appears, click Add Data Source.

  3. In the Add Connection dialog box, configure the parameters.

    Parameter

    Description

    Type

    The type of data source to add.

    Data source name

    The display name of the data source. You can specify a custom name.

    Region

    Select your region.

    Project

    Select the DataWorks workspace.

    AppKey/AppSecret

    The AppKey ID and AppSecret for an account with access permissions to the workspace in DataWorks DataService Studio.

    Note

    You can log on to the DataWorks console and go to the Service Management > Call APIs page to find this information. In the API Call Authentication section, you can obtain the AppKey and AppSecret from Authentication Method 2: Encrypted Signature Authentication.

Create a project

  1. In the DataV console, go to the Projects page, and click PC in the left-side pane.

  2. In the Scenarios section, select a template and click Create Project. This topic uses the Smart Factory Production Dashboard template as an example. Enter a Project Name, select a group from the Select Group drop-down list, and click Create.

    The following steps demonstrate how to replace the template's static data with data from your DataService Studio API.

Configure the project

  1. In the Layer pane on the left, expand Bottom Panel and select the ECharts Basic Line Chart component. In the right-side configuration pane, click the Data Sources tab. For Data Source Type, select DataWorks DataService Studio.

  2. Select the data source and API you just created, and set the query parameters. In this example, set pageSize to 31 to query one month of data.

  3. Click View Data Response to view the API query results.

  4. Configure the field mapping. For x, enter date to use the date as the horizontal axis. For y, enter amount to use the transaction amount as the vertical axis.

    At this point, the x and y fields do not match any data. This is because DataV requires a specific data format and cannot parse deeply nested fields. You need to add a data filter to remove unnecessary fields. In this case, you can simply return the rows array.

  5. Select the Filter checkbox and click the Create button. You can write JavaScript code here to filter or process the data results. The data parameter of the filter is the JSON object returned by the API.

    This example only needs to return the rows array from the API result. Enter return data.data.rows;, preview the filtered result below, and click Save.

  6. After you add the filter, the fields match successfully.

    In the Data Mapping panel, the x field is mapped to date, and the y field is mapped to amount, which indicates that the field mapping is effective.

  7. You have now generated an API from a MaxCompute table in DataService Studio and used it to display data on a DataV dashboard. The ECharts Basic Line Chart component at the bottom of the project now successfully displays the transaction amount data queried through the DataService Studio API, with the date (date) on the horizontal axis and the amount (amount) on the vertical axis.

Usage notes

This seamless integration between DataWorks DataService Studio and DataV means you no longer need to use an API data source in DataV and enter a URL to call an API. Instead, you can create a DataWorks data source and directly select APIs from DataService Studio. This method is fast, convenient, secure, and reliable. It eliminates the need to configure AppKey and AppSecret authentication for each API and supports configuring API parameters in a form.

With DataService Studio, you can directly present the processed data results from MaxCompute in DataV, which provides an end-to-end workflow from data development and data services to data analysis and visualization.

Keep the following in mind during development:

  • In DataWorks DataService Studio, Wizard Mode generates APIs that support only simple conditional queries on a single table. The code editor allows you to write SQL query statements and supports multi-table joins, functions, and complex conditions. You can choose a mode based on your requirements.

  • If you require millisecond-level API query performance, we recommend using a relational database, NoSQL database, or AnalyticDB as your data source.

  • DataV components require data in an array format, but an API generated by DataService Studio returns a complete JSON object that wraps the data array. Therefore, you must use a filter to process the API results. You can add a filter in either DataV or directly in DataService Studio when you configure the API.

    Typically, for an API without pagination, return the data array. For an API with pagination, return the data.rows array.

  • If you want to add multiple series to a line or bar chart in DataV, DataV usually requires the data for each series to be an object and distinguishes series by fields. In this case, you must use a filter to convert the data format.

Note

In addition to providing data services for DataV, DataWorks DataService Studio APIs can be used in various other scenarios, such as BI reporting tools, API marketplaces, data sharing, and programmatic calls.

Previous：Analyze offline data in internet and e-commerce industriesNext：Big data security administration practice guide