Configure API input component

Prerequisites

• You have created an API data source. For more information, see Create an API data source.

• Your account must have read permission on the data source. If you do not have the required permission, apply for it. For more information, see Apply for data source permissions.

Procedure

1. In the top navigation bar of the Dataphin homepage, choose Develop > Data Integration.

2. In the top navigation bar, select a project. In Dev-Prod mode, you must also select an environment.

3. In the left-side navigation pane, click Offline Integration. In the Offline Integration list, click the offline pipeline to open its configuration canvas.

4. In the upper-right corner of the canvas, click Component Library to open the component library panel.

5. In the left-side navigation pane of the component library panel, select Input. Find the API component and drag it onto the canvas.

6. On the API input component card, click to open the API Input Configuration dialog box.

7. In the API Input Configuration dialog box, configure the following parameters.

   Parameter	Description
   Step name	The name of the API input component. Dataphin automatically generates a name, but you can change it. The name must meet the following requirements: Can contain only Chinese characters, letters, digits, and underscores (_). Cannot exceed 64 characters in length.
   Data source	Select an API data source. The list displays all API data sources in the current Dataphin instance, including those with and without your read permission. If you lack read permission for a data source, click Apply next to it to request permission. For more information, see Apply for data source permissions. If no API data source exists, click New to create one. For more information, see Create an API data source.
   Request method	The supported methods are POST and GET. GET is the default. POST: You can optionally configure Params, Headers, and Body. GET: You can optionally configure Params and Headers.
   URL path	Optional. Enter the URL path. This path is appended to the URL from the data source to form the complete request URL. For example, if the data source URL is https://www.aliyun.com and the URL path is /home, the request URL becomes https://www.aliyun.com/home.
   Read JSON path (Optional)	Enter a JSONPath expression, such as data.result, to read specific data from the response. If you leave this parameter empty, the entire response is processed.
   Number of requests	Choose to send a one-time request or multiple requests. One-time request: Sends a single request. Multiple requests: Sends multiple requests based on a specified loop configuration. If you select this option, you must configure the following parameters. Loop mode: Select Pagination loop or Parameter traversal loop. Pagination loop Pagination method: Select Page number/offset or Cursor. Request parameter: Configure the parameter for the loop request. You can select an existing parameter from Parameter or Body, or enter one manually. You can enter values manually only when the Body format is JSON. You can use JSONPath expressions such as a.b.c for requests with a JSON body. Parameter value source (Available only when Pagination method is Cursor): This value must come from a response parameter. Enter the value from the response to use for the next request. JSONPath is supported. Start value (Optional): If Pagination method is Page number/offset: Enter the start value. Default: 1. If Pagination method is Cursor: Enter the start value. Default: 0. Request step (Optional, available only when Pagination method is Page number/offset): Enter the step value. Default: 1. Termination condition: The loop stops when this condition is met. Left condition: Configure the parameter source and value. Parameter source: Select Request parameter, Response parameter, or Number of requests. Default: Response parameter. Parameter value: JSONPath expressions are supported. If you select Request parameter for Parameter source, you do not need to configure the operator or right condition. Operator: The available operators are Is empty, Is not empty, =, ≠, ＞, ≥, ＜, and ≤. When you select Is empty or Is not empty, you do not need to configure a right-hand value. If you select empty, the termination condition is satisfied if the left-hand parameter is null, an empty string, or an empty array (such as data: []). Right condition: Configure the parameter source and value. Parameter source: Select Request parameter, Response parameter, or Constant. Default: Constant. Parameter value: JSONPath expressions are supported. Parameter traversal loop Request parameter: Configure the parameter for the loop request. You can select an existing parameter from Parameter or Body, or enter one manually. You can enter values manually only when the Body format is JSON. You can use JSONPath expressions such as a.b.c for requests with a JSON body. Acquisition method: Manual Entry: In the parameter value list, enter the values for the loop. You can use line breaks to separate values. API Acquisition: This method supports only APIs that do not require authentication. URL: Configure the request URL to retrieve the parameter list. You can select POST or GET as the request method. GET is the default method. Request parameters: Configure additional parameters for the request. This field is empty by default. Click New to add a parameter and configure its name, value, and destination. Parameter name and Parameter value: All characters are supported. Add to: Select Params, Header, or Body. Default: Params. You can select Body only when the request method is POST. Output value: Enter the output value. JSONPath expressions such as data.list are supported. Termination condition: The loop terminates after iterating through all specified parameter values.
   Request parameters	Request parameters include Params, Headers, Body (available only if Request method is POST), and Signature. Params, headers, and body Parameter name: Supports Chinese characters, letters, and special characters. Parameter type: You can select Constant, Timestamp, or Random number. Parameter value: If Parameter type is set to Constant, the parameter value can contain any character. If Parameter type is set to Timestamp, the parameter value is automatically generated. If Parameter type is set to Random number, you can select 16-bit, 32-bit, 64-bit, or 128-bit for the automatically generated random number. Data type: You can configure the data type only in the Body. Supported data types are String, Long, JSON, Double, Boolean, and Date. String is the default data type. Actions: New: Click +New to add a parameter. You must click the icon to save the parameter before you can create another one. Click the icon to cancel adding the parameter. Edit: In the Actions column of a configured parameter, click the icon to edit its settings. Move: In the Actions column of a parameter, click the icon to move it to a different list, for example, from Params to Header or Body. You cannot move a parameter if a parameter with the same name exists in the destination list, or if the Body input method is JSON. Delete: In the Actions column of a configured parameter, click the icon to delete the parameter. You cannot delete a parameter if a signature configuration references it. The input format for the Body can be JSON or Key-Value. JSON: Enter the JSON code in the code editor. Click the icon to automatically format the code. Key-Value: Click New, enter the parameter name and value, and then select the parameter type and data type. Signature The signature feature is disabled by default. To enable it, you must configure the signature generation rules. Signature name: The name of the signature parameter added to the request. Signature location: Select Params, Header, or Body (available only if Request method is POST). Generation function: Select MD5HEX, HMAC-MD5, SHA1HEX, HMAC-SHA256, HMAC-SHA1, SHA256, SHA256HEX, SHA512HEX, or HMAC_SHA512. Secret key: Select a secret key from Governance > Data Security > Key Management, or enter one manually. If you have not purchased the Asset Security add-on for your Dataphin instance, you can only enter the secret key manually. If you have purchased the Asset Security add-on, you must select a secret key from Governance > Data Security > Key Management. If no secret keys are available, click Go to Register to go to the Governance > Data Security > Key Management page and register one. Note This parameter is optional if the Generation function is set to MD5HEX, SHA1HEX, SHA256, SHA256HEX, or SHA512HEX. Concatenation content: Select Parameter value only, Parameter name + parameter value, or Custom. Default: Parameter name + parameter value. Parameter concatenation Referenced parameters: Select one or more key-value parameters from the parameter lists. Concatenation order: Select Dictionary order or Selection order. Default: Dictionary order. Delimiters (Optional): Configure a parameter separator and a key-value delimiter. Parameter separator: Enter a custom separator for parameters. Default: &. Key-value delimiter: Enter a custom delimiter for key-value pairs. Default: =. You cannot configure this parameter if Concatenation content is set to Parameter value only. Concatenation example: Displays the concatenated string based on your configuration. If a parameter uses a timestamp or random number, a temporarily generated value is shown. Custom Signature string: Enter a custom signature string. Type @ to trigger parameter suggestions. If you switch to Custom mode from another mode that has referenced parameters, the system automatically concatenates them and populates the signature string based on the specified order and delimiters. You can include the secret key, constants, and parameters from Params, Header, or Body. If you select Secret key, #{secret} is added to the string. If you select All Params, All Header, or All Body, all parameters from the selected list are added to the string in K:V format. For example, a parameter named id is added as id:#{id}. If the Body is in JSON mode, selecting it inserts the full JSON content, represented by the #{body_json} placeholder. Case: Select Keep default format of signature function, Convert signature value to uppercase, or Convert signature value to lowercase. The default option is 'Keep default format of signature function'. The system converts the case of the signature value after the function generates it, without changing the original content used in the calculation.
   Output fields	This section displays the output fields. You can also add them manually. Batch-add output fields. Format: Click Batch Add. You can configure fields in JSON or TEXT format. JSON format: // Example: [{ "index": "0", "name": "user_id", "type": "String" }, { "index": "1", "name": "user_name", "type": "String" }] Note index specifies the column number, name specifies the field name after import, and type specifies the field type. For example, "index":3,"name":"user_id","type":"String" imports the fourth column from the source, renames the field to user_id, and sets its type to String. TEXT format: // Example: 0,user_id,String 1,user_name,String Row delimiter and Column delimiter: When adding fields in TEXT format, you can configure row and column delimiters. The row delimiter separates field information. The default is a newline character (\n). You can also use \n ; . The column delimiter separates the field name and type. The default is a comma (,). Add a single output field. Click New Output Field, enter the Source Index and Field name, and select a Type. Click the icon to save the row, or the icon to delete it. Manage output fields. You can perform the following actions on added fields: Click the icon in the Actions column to edit a field. Click the icon in the Actions column to delete a field.

8. Click Confirm to save the component's configuration.