DocsICP FilingConsole
Documentation
Home Page Data Management User Guide Data+AI Data warehouse development Data warehouse development task flows

Data warehouse development task flows

Updated at:
Copy as MD
Product
My Favorites

A workspace for data warehouse development consists of data warehouse solutions and business scenarios. You can create multiple business scenarios in a workspace and add multiple task flows to each business scenario. This lets you perform complex data processing and task scheduling.

Prerequisites

A workspace is created. For more information, see Create a workspace.

Create a data warehouse development task flow

  1. Log in to DMS 5.0.

  2. In the top navigation bar, choose Data+AI > Data Development > Data Warehouse Development.

    Note

    If you use the DMS console in simple mode, move the pointer over the 2023-01-28_15-57-17.png icon in the upper-left corner and choose All Features > Data+AI > Data Development > Data Warehouse Development.

  3. In the Workspace section, click the name of the target workspace.

  4. Optional: Create a business scenario.

    1. In the navigation pane on the left of the workspace, choose jai > Business Scenario.

    2. In the Create Business Scenario dialog box, enter a name and description for the business scenario.

    3. Click Confirm.

    Note

    The system automatically creates a default business scenario. You can right-click the business scenario to rename or delete it, or create a new task flow in it.

  5. Create a new task flow.

    1. Right-click the target business scenario and select New Task Flow.

    2. In the New Task Flow dialog box, enter a name and description for the task flow.

    3. Click Confirm.

  6. On the task flow details page, edit, test, and publish the task flow.

    Job stream

    Description

    References

    Data integration

    • Displays the data integration nodes in the task flow: Data Transmission Service (DTS) data migration nodes and offline data integration nodes.

    • If you test a data integration node in a workspace in standard mode, the node is dry-run. This prevents the risk of migrating production data to the staging environment.

    Data Development

    • Displays the data development nodes in the task flow. These include all nodes except data integration nodes.

    • The scheduling feature for data development nodes is the same for trial runs and after publishing.

    Table

    The task flow can reference and operate on data warehouse tables. You can create, import, edit, and remove these tables.

    Manage tables in a task flow

    The operations for task flows in the Data Warehouse Development feature are similar to those for task flows in the Task Orchestration feature. For more information, see Overview.

Manage tables in a task flow

  1. Log in to DMS 5.0.

  2. In the top navigation bar, choose Data+AI > Data Development > Data Warehouse Development.

    Note

    If you use the DMS console in simple mode, move the pointer over the 2023-01-28_15-57-17.png icon in the upper-left corner and choose All Features > Data+AI > Data Development > Data Warehouse Development.

  3. In the Workspace section, click the name of the target workspace.

  4. In the navigation pane on the left, click the 5展开3 icon next to the target business scenario to expand its task flows.

  5. Click the 5展开2 icon next to the target task flow to expand its subdirectories.

  6. Manage data warehouse tables.

    Actions

    Description

    Procedure

    Create a table

    • A table is created in both the staging and production databases using the same table schema (the same CREATE TABLE statement).

    • The data warehouse development task flow can then operate on this data warehouse table.

    1. Right-click Table and select New Table.

    2. In the New Table section, create the data warehouse table as prompted.

    Important

    Log on to both the staging and production databases before you create a table.

    Import a table

    • The production and staging databases must contain tables with the same name and schema.

      Note

      If the target table does not exist in the staging database, you can use the schema synchronization feature to sync the table to the staging database. For more information, see Schema synchronization.

    • The data warehouse development task flow can then operate on the imported data warehouse table.

    1. Right-click Table and select Import Table.

    2. In the Import Data Warehouse Table dialog box, select the target table, subject, and level to import, and then click Confirm.

    Important

    Log on to both the staging and production databases before you import a table.

    Edit a table

    Edits apply to the table in both the production and staging databases.

    1. Right-click the target data warehouse table and select Edit.

    2. On the Table Design page, edit the table.

    Remove a table

    • This operation only removes the reference to the data warehouse table from the task flow. It does not delete the physical table from the database.

    • After the table is removed, the data warehouse development task flow can no longer operate on it.

    1. Right-click the target data warehouse table and select Delete.

    2. In the Remove Associated Table dialog box, click Confirm.

Previous：Data ingestion solutionsNext：Configure a DTS checkpoint node