Configure the MaxCompute Output Component

Procedure

1. In the top menu bar on the Dataphin homepage, click Develop > Data Integration.

2. On the Integration page, select a Project in the top menu bar. In Dev-Prod mode, also select an environment.

3. In the navigation pane on the left, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline that you want to develop. The pipeline configuration page opens.

4. In the upper-right corner of the page, click Component Library to open the Component Library panel.

5. In the navigation pane on the left of the Component Library panel, click Output. In the output component list on the right, find the MaxCompute component and drag it onto the canvas.

6. Click and drag the icon of your input component to connect it to the MaxCompute output component.

7. Click the icon in the MaxCompute output component card to open the MaxCompute Output Configuration dialog box.

8. In the MaxCompute Output Configuration dialog box, configure the parameters.

   Parameter		Description
   Basic Settings	Step Name	The component name. Dataphin generates a default name that you can change. Naming rules: Use only Chinese characters, letters, underscores (_), and digits. Keep the name under 64 characters.
   	Datasource	Lists all MaxCompute data sources, including those you have sync-write permissions for and those you do not. Click the icon to copy the data source name. If you lack sync-write permissions for a data source, click Request next to the data source to request sync-write permissions. For details, see Request, renew, or release data source permissions. If you do not have any MaxCompute data sources, click Create Data Source to create one. For details, see Create a MaxCompute data source.
   	Table	Select the target table for output data. If no target table exists in your MaxCompute data source, use the One-click Generate Target Table feature to create one quickly. Follow these steps: Click One-click Create Table. Select Standard Table or Delta Table. Standard Table is selected by default. Switching table formats updates the DDL in the code area below. Dataphin auto-generates the SQL script with the table name (defaulting to the source table name) and field types (converted from Dataphin fields). Adjust as needed, then click Create. Dataphin then uses the new table as the output target.
   	Production Table Missing Policy	Specifies what happens when the production table does not exist. Select Take No Action or Automatic Creation (default: Automatic Creation). No Action: If the target table does not exist, the system warns you at submission time but still allows publishing. You must manually create the target table in the production environment before running the task. Automatic Creation: You need to Edit Table Creation Statement. The statement is pre-filled with the table creation statement of the selected table, and you can adjust it. The table name in the statement uses the placeholder ${table_name}, and only this placeholder is supported. It will be replaced with the actual table name during execution. If the target table does not exist, Dataphin first runs the table-creation statement. If creation fails, publishing fails. Fix the statement using the error message, then republish. If the target table already exists, no creation occurs. Note This setting applies only to projects in Dev-Prod mode.
   	Loading Policy	Specifies how data is written to the target table. Overwrite: Replace historical data in the target table with data from the current source table. Append: Add new data to existing data in the target table without changing historical data. Available only for standard tables. Update: Update mapped fields in existing records when primary key conflicts occur. Available only for Delta tables.
   	Partition	If you select a partitioned table, enter the partition. Examples: ds=20230101 or ds=${bizdate}. You can add the /*dynamic*/ keyword before a partition to dynamically specify the write partition for each row of data based on source fields. For example, /*dynamic*/ds=$date, where date is the name of a source field; or /*dynamic*/ds=${bizdate},hh = $hour, where bizdate is a parameter and hour is the field name in the source table. Important Dynamic partitions are capped at 10,000. We recommend no more than 1,000 to avoid long run times or failures.
   	Preparation Statement (optional)	An SQL script that runs before data import. For example, for zero-downtime updates: create Target_A, write data to it, rename Service_B to Temp_C, rename Target_A to Service_B, then delete Temp_C.
   	Finalization Statement (optional)	An SQL script that runs after data import.
   Field Mapping	Input Fields	Shows input fields from upstream components.
   	Output Fields	Lists all fields in the selected table. To exclude fields from downstream processing, remove them: To remove individual fields, click the icon in the Actions column. To remove many fields, click Field Management. In the Field Management page, select multiple fields, then click the icon to move Selected Input Fields to Unselected Input Fields.
   	Mapping	Map input fields from the source table to output fields in the target table. Two mapping types are supported: Name-based mapping: Maps fields with identical names. Row-based mapping: Maps fields by position when source and target field names differ.

9. Click Confirm to finish configuring the MaxCompute output component.