The service orchestration feature in DataService Studio provides a visual, drag-and-drop interface to build workflows. You can arrange multiple APIs and functions into a workflow with sequential, parallel, and branching structures to match your business logic.
Prerequisites
-
The service orchestration feature requires DataWorks Enterprise Edition or a later version. For more information, see Editions and billing.
-
The service orchestration feature is available only in the China (Shanghai) region. This feature is currently undergoing an upgrade and is temporarily unavailable. We appreciate your patience.
Background information
When you call a Service arrangement API service, the system executes the configured service nodes in sequence, passes parameters between them, and automatically manages their state transitions. The service orchestration feature greatly simplifies the development and O&M costs of combining multiple services, allowing you to focus on your business logic.
Service orchestration offers the following advantages:
-
Reduced API development costs
Build workflows with a visual, drag-and-drop interface. You can arrange API calls in sequential, parallel, or branching structures without writing extra code, significantly reducing development costs.
-
Improved service call performance
Executing multiple API or function calls within the same container instance reduces network overhead compared to building your own workflow service, leading to a significant improvement in performance.
-
Serverless architecture
Service orchestration uses a serverless architecture with auto scaling. You do not need to manage the underlying runtime environment, so you can focus solely on your business logic.
Input and output rules
DataService Studio uses JSONPath to retrieve parameter values. JSONPath is an expression language for extracting data from a JSON document. For more information, see the official JSONPath documentation.
For example, consider a sequence of three nodes: A, B, and C. Node C needs to access the output from both Node A and Node B:
-
Output of Node A: {"namea":"valuea"}.
To access the output of Node A, use ${A.namea}.
-
Output of Node B: {"nameb":"valueb"}.
To access the output of Node B, use $.nameb or ${B.nameb}.
The system includes a built-in start node that provides the initial input parameters for the workflow. For example, if a workflow's input is {"namewf":"valuewf"}, any node can access this value using the expression ${START.namewf}.
The start node and end node are built-in system nodes. You can rename them, but you cannot delete them. The start node is the initial step, or Node 0, of the workflow.
Parameters
-
Workflow request parameters
Click Request Parameters on the right side of the service orchestration edit page to configure the request parameters by selecting Add manually or Automatic analysis:
-
Add manually: Click Add Parameter to add request parameters for the workflow.
-
Automatic analysis: If the first node of the workflow is an API node, click Automatically parse request parameters to map the API's request parameters to the workflow.
-
-
API node request parameters
Click an API node, and in the panel that appears, click Input Request Parameters to configure its request parameter values.
-
By default, the system performs a name-based mapping, matching the parameter with a field of the same name at the top level of the previous node's JSON output.
NoteIf the current node is the first node in the workflow, its parameters are mapped by name from the workflow's main request parameters.
-
If you set a parameter value, the system uses the value you provide.
NoteTo set the parameter value of the current node to a specific parameter value from a designated upstream node, you must use a JSONPath expression.
-
-
API node response parameters
Click an API node, select the set output results checkbox, and use a JSONPath expression to customize the output. For example:
{ "return1":"$.data.rows.user_id", "return2":"$.data.rows.user_name" } -
Python node request parameters
Click a Python node and enter the content for the request parameters in the Request Parameters editor.
-
Python node response parameters
Click a Python node, select the set output results checkbox, and use a JSONPath expression to customize the output. For example:
{ "return1":"$.data.rows.user_id", "return2":"$.data.rows.user_name" }
The following table describes common JSONPath expressions for retrieving parameter values.
|
JSONPath expression |
Request usage |
Response usage |
|
$. |
References the root object of the previous node's output. |
References the root object of the current node's output. |
|
$.param |
Retrieves the value of the param parameter from the previous node's output. |
Retrieves the value of the param parameter from the current node's output. |
|
${START} |
Retrieves the request parameters of the workflow, which is the output of the start node. |
|
|
${NodeID} |
Retrieves the output of the node with the specified ID. |
|
|
${NodeID.param} |
Retrieves the value of the param parameter from the output of the node with the specified ID. |
|
Example
Log on to the DataWorks console. In the target region, click in the left-side navigation pane. Select a workspace from the drop-down list and click Go to DataService Studio.
-
Register an API.
This example registers an API from an existing endpoint.
-
In the navigation pane on the left, click the
icon to go to the Service Development page. -
Hover over the
icon and choose . -
In the Register API dialog box, configure the parameters. For more information, see Register an external API service.
-
Click Determine.
-
-
Register a function.
-
On the Service Development page, hover over the
icon and choose .Alternatively, you can find the relevant business process, right-click Function, and choose .
-
In the Create Python Function dialog box, configure the parameters. For more information, see Manage functions.
-
Click OK.
-
On the function's configuration page, enter the following code in the Edit Code area.
# -*- coding: utf-8 -*- # event (str) : in filter it is the API result, in other cases, it is your param # context : some environment information, temporarily useless # import module limit: json,time,random,pickle,re,math import json def handler(event,context): # load str to json object obj = json.loads(event) # add your code here # end add return obj -
In the Environment Configuration section, set the Memory and Timeout.
-
Click the
icon in the toolbar.
-
-
Create a workflow.
-
On the Service Development page, hover over the
icon and click Create Workflow.Alternatively, you can find the relevant business process, right-click Service arrangement, and choose Create Workflow.
-
In the Service arrangement dialog box, configure the parameters.
Parameter
Description
API Name
Must be 4 to 50 characters in length and can contain Chinese characters, letters, digits, and underscores (_). It must start with a Chinese character or a letter.
APIPath
The path where the API is stored, for example, /user.
NoteThe path can be up to 200 characters long and can contain letters, digits, underscores (_), and hyphens (-). It must start with a forward slash (/).
Protocol
Supported protocols are HTTP and HTTPS.
To call the API over HTTPS, publish the API to the gateway, bind an independent domain name, and upload an SSL certificate in the API Gateway console. For more information, see Enable HTTPS support.
Request Method
Supported methods are GET and POST.
Response Type
Only the JSON response type is supported.
Visibility Scope
Options include Workspace and Private:
-
Workspace: The API is visible to all members within this workspace.
-
Private: The API is visible only to the API owner. Authorizing other users is not currently supported.
NoteIf visibility is set to Private, only you can see the API in the directory tree; other members of the workspace cannot.
Tag
Select appropriate tags from the Tag list. For more information, see Create and manage API tags.
NoteTag names can contain Chinese characters, letters, digits, and underscores (_). You can add up to five tags, and each tag can be up to 20 characters long.
Description
Enter a brief description of the API, up to 2,000 characters.
Location
The directory where the service orchestration workflow is stored.
-
-
Click OK.
-
-
Edit the workflow.
-
On the workflow editor page, drag and connect the corresponding modules in the order shown below.
The nodes, from top to bottom, are Start, API, SWITCH, PYTHON, and End. Drag and connect these nodes on the canvas in that order.
-
Click the API1 node to edit it. From the Select API list, choose the API you registered. Select the set output results checkbox and enter
{"user_id":"$.data[0].id"}.The output is processed using JSONPath. The syntax for retrieving node values (for example, ${NodeA.namea}) is the same as for input parameters. The expression
{"user_id":"$.data[0].id"}assigns the id of the first element in the data array of the current node's result to user_id. The node then outputs JSON data in the format {"user_id":"value"}. -
Click the PYTHON1 node, and from the Select Function list, choose the function you registered.
-
Click the SWITCH1 node, and in the editor dialog, click Set branch conditions.
The conditional expression evaluates the output of the previous node. For example,
${NodeID.OutputName}>1or$.OutputName>1. The supported operators include ==, !=, >=, >, <=, <, &&, !, (), +, -, *, /, and %.In this example, the user_id output from the upstream node is used as the request parameter for the downstream node.
New Branch 1: $.user_id != 1 # Executes if the upstream user_id is not 1. New Branch 2: $.user_id == 1 # Executes if the upstream user_id is 1. -
Click the Complete node, then click Response Parameters on the right side of the page to configure the response parameters.
-
-
Test the workflow.
-
Click Test in the upper-right corner of the page.
-
In the Test APIs dialog box, click Determine.
-
View the Runtime Log and Execution results at the bottom of the page.
-