Service orchestration

更新时间:
复制 MD 格式

The service orchestration feature in DataService Studio provides a visual, drag-and-drop interface to build workflows. You can arrange multiple APIs and functions into a workflow with sequential, parallel, and branching structures to match your business logic.

Prerequisites

  • The service orchestration feature requires DataWorks Enterprise Edition or a later version. For more information, see Editions and billing.

  • The service orchestration feature is available only in the China (Shanghai) region. This feature is currently undergoing an upgrade and is temporarily unavailable. We appreciate your patience.

Background information

When you call a Service arrangement API service, the system executes the configured service nodes in sequence, passes parameters between them, and automatically manages their state transitions. The service orchestration feature greatly simplifies the development and O&M costs of combining multiple services, allowing you to focus on your business logic.

Service orchestration offers the following advantages:

  • Reduced API development costs

    Build workflows with a visual, drag-and-drop interface. You can arrange API calls in sequential, parallel, or branching structures without writing extra code, significantly reducing development costs.

  • Improved service call performance

    Executing multiple API or function calls within the same container instance reduces network overhead compared to building your own workflow service, leading to a significant improvement in performance.

  • Serverless architecture

    Service orchestration uses a serverless architecture with auto scaling. You do not need to manage the underlying runtime environment, so you can focus solely on your business logic.

Input and output rules

DataService Studio uses JSONPath to retrieve parameter values. JSONPath is an expression language for extracting data from a JSON document. For more information, see the official JSONPath documentation.

For example, consider a sequence of three nodes: A, B, and C. Node C needs to access the output from both Node A and Node B:

  • Output of Node A: {"namea":"valuea"}.

    To access the output of Node A, use ${A.namea}.

  • Output of Node B: {"nameb":"valueb"}.

    To access the output of Node B, use $.nameb or ${B.nameb}.

The system includes a built-in start node that provides the initial input parameters for the workflow. For example, if a workflow's input is {"namewf":"valuewf"}, any node can access this value using the expression ${START.namewf}.

Note

The start node and end node are built-in system nodes. You can rename them, but you cannot delete them. The start node is the initial step, or Node 0, of the workflow.

Parameters

  • Workflow request parameters

    Click Request Parameters on the right side of the service orchestration edit page to configure the request parameters by selecting Add manually or Automatic analysis:

    • Add manually: Click Add Parameter to add request parameters for the workflow.

    • Automatic analysis: If the first node of the workflow is an API node, click Automatically parse request parameters to map the API's request parameters to the workflow.

  • API node request parameters

    Click an API node, and in the panel that appears, click Input Request Parameters to configure its request parameter values.

    • By default, the system performs a name-based mapping, matching the parameter with a field of the same name at the top level of the previous node's JSON output.

      Note

      If the current node is the first node in the workflow, its parameters are mapped by name from the workflow's main request parameters.

    • If you set a parameter value, the system uses the value you provide.

      Note

      To set the parameter value of the current node to a specific parameter value from a designated upstream node, you must use a JSONPath expression.

  • API node response parameters

    Click an API node, select the set output results checkbox, and use a JSONPath expression to customize the output. For example:

    {
      "return1":"$.data.rows.user_id",
      "return2":"$.data.rows.user_name"
    }
  • Python node request parameters

    Click a Python node and enter the content for the request parameters in the Request Parameters editor.

  • Python node response parameters

    Click a Python node, select the set output results checkbox, and use a JSONPath expression to customize the output. For example:

    {
      "return1":"$.data.rows.user_id",
      "return2":"$.data.rows.user_name"
    }

The following table describes common JSONPath expressions for retrieving parameter values.

JSONPath expression

Request usage

Response usage

$.

References the root object of the previous node's output.

References the root object of the current node's output.

$.param

Retrieves the value of the param parameter from the previous node's output.

Retrieves the value of the param parameter from the current node's output.

${START}

Retrieves the request parameters of the workflow, which is the output of the start node.

${NodeID}

Retrieves the output of the node with the specified ID.

${NodeID.param}

Retrieves the value of the param parameter from the output of the node with the specified ID.

Example

  1. Log on to the DataWorks console. In the target region, click Data Analysis and Service > DataService Studio in the left-side navigation pane. Select a workspace from the drop-down list and click Go to DataService Studio.

  2. Register an API.

    This example registers an API from an existing endpoint.

    1. In the navigation pane on the left, click the image icon to go to the Service Development page.

    2. Hover over the image icon and choose Create API > Register API.

    3. In the Register API dialog box, configure the parameters. For more information, see Register an external API service.

    4. Click Determine.

  3. Register a function.

    1. On the Service Development page, hover over the 新建 icon and choose Function > Create Python Function.

      Alternatively, you can find the relevant business process, right-click Function, and choose Create > Create Python Function.

    2. In the Create Python Function dialog box, configure the parameters. For more information, see Manage functions.

    3. Click OK.

    4. On the function's configuration page, enter the following code in the Edit Code area.

      # -*- coding: utf-8 -*-
      # event (str) : in filter it is the API result, in other cases, it is your param
      # context : some environment information, temporarily useless
      # import module limit: json,time,random,pickle,re,math
      import json
      def handler(event,context):
          # load str to json object
          obj = json.loads(event)
          # add your code here
          # end add
          return obj
    5. In the Environment Configuration section, set the Memory and Timeout.

    6. Click the 保存 icon in the toolbar.

  4. Create a workflow.

    1. On the Service Development page, hover over the 新建 icon and click Create Workflow.

      Alternatively, you can find the relevant business process, right-click Service arrangement, and choose Create Workflow.

    2. In the Service arrangement dialog box, configure the parameters.

      Parameter

      Description

      API Name

      Must be 4 to 50 characters in length and can contain Chinese characters, letters, digits, and underscores (_). It must start with a Chinese character or a letter.

      APIPath

      The path where the API is stored, for example, /user.

      Note

      The path can be up to 200 characters long and can contain letters, digits, underscores (_), and hyphens (-). It must start with a forward slash (/).

      Protocol

      Supported protocols are HTTP and HTTPS.

      To call the API over HTTPS, publish the API to the gateway, bind an independent domain name, and upload an SSL certificate in the API Gateway console. For more information, see Enable HTTPS support.

      Request Method

      Supported methods are GET and POST.

      Response Type

      Only the JSON response type is supported.

      Visibility Scope

      Options include Workspace and Private:

      • Workspace: The API is visible to all members within this workspace.

      • Private: The API is visible only to the API owner. Authorizing other users is not currently supported.

        Note

        If visibility is set to Private, only you can see the API in the directory tree; other members of the workspace cannot.

      Tag

      Select appropriate tags from the Tag list. For more information, see Create and manage API tags.

      Note

      Tag names can contain Chinese characters, letters, digits, and underscores (_). You can add up to five tags, and each tag can be up to 20 characters long.

      Description

      Enter a brief description of the API, up to 2,000 characters.

      Location

      The directory where the service orchestration workflow is stored.

    3. Click OK.

  5. Edit the workflow.

    1. On the workflow editor page, drag and connect the corresponding modules in the order shown below.

      The nodes, from top to bottom, are Start, API, SWITCH, PYTHON, and End. Drag and connect these nodes on the canvas in that order.

    2. Click the API1 node to edit it. From the Select API list, choose the API you registered. Select the set output results checkbox and enter {"user_id":"$.data[0].id"}.

      The output is processed using JSONPath. The syntax for retrieving node values (for example, ${NodeA.namea}) is the same as for input parameters. The expression {"user_id":"$.data[0].id"} assigns the id of the first element in the data array of the current node's result to user_id. The node then outputs JSON data in the format {"user_id":"value"}.

    3. Click the PYTHON1 node, and from the Select Function list, choose the function you registered.

    4. Click the SWITCH1 node, and in the editor dialog, click Set branch conditions.

      The conditional expression evaluates the output of the previous node. For example, ${NodeID.OutputName}>1 or $.OutputName>1. The supported operators include ==, !=, >=, >, <=, <, &&, !, (), +, -, *, /, and %.

      In this example, the user_id output from the upstream node is used as the request parameter for the downstream node.

      New Branch 1: $.user_id != 1   # Executes if the upstream user_id is not 1.
      New Branch 2: $.user_id == 1   # Executes if the upstream user_id is 1.
    5. Click the Complete node, then click Response Parameters on the right side of the page to configure the response parameters.

  6. Test the workflow.

    1. Click Test in the upper-right corner of the page.

    2. In the Test APIs dialog box, click Determine.

    3. View the Runtime Log and Execution results at the bottom of the page.