Configure transformation components

Limitations

Prerequisites

You have configured the data source.

Configure JOIN

1. Go to the Streaming ETL page.

   1. Log in to the DTS console.

   2. In the left-side navigation pane, click ETL.

2. In the upper-left corner, click the icon. In the Create Data Flow dialog box, enter a name in the Data Flow Name field, and set Development Method to DAG.

3. Click Confirm.

4. Configure the source databases. For more information, see Configure a source database.

   Important

   You need to configure two source databases.

5. In the left-side pane, drag the Table Join node to a blank area on the canvas.

6. Hover over a configured data source node, click the connection point, and drag a line to the JOIN-1 node.

7. On the canvas, click the JOIN-1 node and configure the transformation component.

   1. On the Node Settings tab, configure the following parameters.

      Section	Parameter	Description
      Transformation name	Enter a transformation name.	A descriptive name is recommended for easy identification. The name does not have to be unique.
      Join Settings	Left table in JOIN clause	Select the table that appears on the left side of the JOIN clause.
      	Temporal join time attribute	Select the time attribute for the temporal join. Based on Event Time Watermark Based on Processing Time Note A temporal join requires a time attribute for each stream table and a primary key on the right table. If the right table is a dimension table, its primary key must be used in an equality condition for the join.
      	Select join operation	Select a join operation. Inner Join: Returns only the matching records from both tables. Left Join: Returns all records from the left table and the matching records from the right table. Right Join: Returns all records from the right table and the matching records from the left table.
      JOIN Condition	+ Add Condition	Click + Add Condition to select fields for the join condition. Note The field to the left of the equals sign (=) is from the left table, and the field to the right is from the right table.

   2. After you configure the join conditions, click the Output Fields tab. In the Column Name column, select the fields to include in the output.

Configure Field Calculator

1. Go to the Streaming ETL page.

   1. Log in to the DTS console.

   2. In the left-side navigation pane, click ETL.

2. In the upper-left corner, click the icon. In the Create Data Flow dialog box, enter a name in the Data Flow Name field, and set Development Method to DAG.

3. Click Confirm.

4. Configure the source database. For more information, see Configure a source database.

5. In the left-side pane, drag the Field Calculator node to a blank area on the canvas.

6. Hover over the configured data source node, click the connection point, and drag a line to the Field Calculator-1 node.

7. On the canvas, click the Field Calculator-1 node and configure the transformation component.

   1. Enter a Conversion Name.

      Note

      A descriptive name is recommended for easy identification. The name does not have to be unique.

   2. In the Calculator Settings section, click + Add Field.

   3. In the configuration list, find the newly added field and click Calculator Settings in the Actions column.

   4. In the Value Settings section, configure the following parameters.

      Parameter	Description
      Field name	DTS automatically generates a name, but a descriptive name is recommended for easy identification. The name does not have to be unique.
      Select ETL data type	Select the appropriate ETL data type.
      Destination field: Field name, value settings	Configure the destination field's calculation logic using one of the following methods: Directly enter the calculation formula. Build the formula by clicking options in the Input Fields, Function Set, or Operator sections.

Configure Table Record Filter

1. Go to the Streaming ETL page.

   1. Log in to the DTS console.

   2. In the left-side navigation pane, click ETL.

2. In the upper-left corner, click the icon. In the Create Data Flow dialog box, enter a name in the Data Flow Name field, and set Development Method to DAG.

3. Click Confirm.

4. Configure the source database. For more information, see Configure a source database.

5. In the left-side pane, drag the Table Record Filter node to a blank area on the canvas.

6. Hover over the configured data source node, click the connection point, and drag a line to the Table Record Filter-1 node.

7. On the canvas, click the Table Record Filter-1 node and configure the transformation component.

   1. Enter a Conversion Name.

      Note

      A descriptive name is recommended for easy identification. The name does not have to be unique.

   2. In the WHERE Condition section, build the condition in one of the following ways:

      • Directly enter the WHERE condition.

      • Build the condition by clicking options in the Input Fields or Operator sections.