Create a script-based integration task

Prerequisites

The source and target data sources are configured. For supported data sources, see Data integration supported data sources.

Procedure

Step 1: Create an offline script

1. On the Dataphin home page, choose Development > Data Integration from the top menu bar.

2. Open the Create Offline Script dialog box:

   Select a project (in Dev-Prod mode, also select the environment) > Click Batch Pipeline > Click the New icon > Click Batch Script.

3. Configure the parameters in the Create Offline Script dialog box.

   Area	Parameter	Description
   Basic Information	Task Name	Name the script. Naming rules: All characters except | : ? <> * " / and \. Maximum 64 characters.
   	Schedule Type	Select the Schedule Type: Recurring Task Node: Periodic execution. Manual Node: Manual execution, no dependencies.
   	Description	Brief description of the script. Maximum 1000 characters.
   	Select Directory	Defaults to the offline pipeline directory. You can create and select a different folder.
   Datasource Config	Source Type	Select the source type.
   	Datasource	Select the source. If unlisted, click Create. Data integration supported data sources. Note Only data sources with read-through permissions are listed. Request data source permission.
   	Target Type	Select the target type.
   	Datasource	Select the target. If unlisted, click Create. Data integration supported data sources. Note Only data sources with write-through permissions are listed. Request data source permission.

4. Click OK.

Step 2: Develop the offline script

Develop the script using the code editor. The JSON script provides flexible data synchronization capabilities with fine-grained configuration control.

Step 3: Configure the pipeline schedule

1. Click on the development canvas menu bar to open schedule configuration.

2. Configure the following settings for the integration pipeline:

   • Basic Information: Set the development and operation owners and add a description. Configure basic information of offline integration pipeline.

   • Schedule Configuration: Define the scheduling type, cycle, logic, and execution properties. Configure offline pipeline schedule.

   • Schedule Dependency: Define dependency nodes. Dataphin executes nodes sequentially based on these dependencies. Configure offline pipeline schedule dependency.

   • Run Configuration: Set task-level run timeout and rerun policies for failed tasks. Offline pipeline task run configuration.

   • Resource Configuration: Assign the task to a resource group for scheduling. Configure offline integration pipeline task resources.

3. Click OK.

Step 4: Save and submit the offline integration task

1. Click the icon at the top of the canvas to save the task.

2. Click the icon to submit the task. In the Submit Remarks dialog box, enter remarks and click OK And Submit.

   Dataphin performs lineage analysis and submission checks upon submission. Integration task submission instructions.

What to do next

• In Dev-Prod mode, publish the task. Manage publishing tasks.

• In Basic mode, the task is scheduled in production upon successful submission. View published tasks in the Operation Center. .