Create an offline dataset through table mapping

Prerequisites

You need to create a tag project to which the dataset belongs before creating an offline dataset. For more information, see Create a tag project.

Procedure

1. On the Dataphin homepage, click Tag > Tag Workbench in the top navigation bar.

2. In the top navigation bar, select Project.

3. In the left-side navigation pane, choose Data Preparation > Offline Dataset.

4. On the Offline Dataset page, click Create Dataset. In the Create Offline Dataset dialog box, select Table Mapping.

5. On the Create Table Mapping configuration page, configure Basic Information, Processing Logic, and O&M Configuration for the dataset.

   • Basic Information

     Parameter	Description
     Dataset Name	Enter the name of the dataset. The name can contain Chinese characters, letters, digits, and underscores (_) and can be up to 64 characters in length.
     Dataset Code	The unique identifier of the offline dataset. It helps you locate a specific offline dataset when there are multiple offline datasets with the same name. The code must start with a letter and can contain lowercase letters, digits, and underscores (_). It can be up to 64 characters in length.
     Dataset Update Method	Supports Periodic Update and Manual Update methods. Periodic update: The dataset is automatically updated at specific intervals. Manual update: The dataset is updated through manual operations.
     Owner	Select the owner of the offline dataset.
     Description	Enter a brief description of the offline dataset. The description can be up to 1,000 characters in length.

   • Processing Logic

     Parameter	Description
     Project/Data Domain	Select the Project or Data version that the offline dataset needs to reference. The dropdown list includes all projects (with bound offline computing sources) and data domains in the current tenant. Note If you have not purchased the Intelligent Development edition, you can only select Project.
     Logical Table/Source Table	Select the logical table/source table that you need to define for the dataset. Logical table: If you selected Data domain in Project/Data domain, you can select a source table. For logical tables, you can only select logical tables with read-through permission. When selecting a logical table, first select the Logical table type, then select the Subject area, and finally select the target Logical table from all logical tables in that subject area. Both subject area and logical table support keyword search. Logical table types include Logical fact table, Logical dimension table, and Logical aggregate table. Note The logical table output method does not include associations by default. Source table: If you selected Project in Project/Data domain, you can select a source table. For source tables, you can only select tables that the project tenant account has permission to query. If you do not have permission, you can click Request Permission to apply. Note Currently, only partitioned tables are supported.
     Date Partition	Select the partition field of the source table. When the source table is a partitioned table, the system will use the field name as the date partition by default. If the default field name is not in the partition field list of the source table, the system will use the first partition field of the table as the date partition.
     Partition Field Format	Enter a date format or select an existing date format. You can select yyyymmdd, yyyy-mm-dd, yyyy/mm/dd, or yyyy.mm.dd.
     Entity ID-Value Type	Select the entity ID field. It will be used for automatic entity ID mapping during tag processing. Note The system will automatically generate the value type based on the entity ID field type.
     Metric Configuration	After completing the selection of Project, Source table, and Entity ID-Value type, you can check the metrics to define the dataset in the metric configuration list, and configure the corresponding Source Field, lookup table, and description. Note The metric name cannot be the same as the First-level Partition Field Name. Partition fields and special field types are not supported for selection. Please refer to whether they are selectable in the actual system interface. Fields that have been used as entity IDs cannot be used as metrics. Search metrics: You can search for metrics by entering the metric name or metric description. Batch add metrics: Click Batch add metrics, and in the Select Source Fields dialog box, you can batch select source fields to add to the Metric Configuration list. Configure lookup table: Only Integer, Decimal(M,0), Boolean, and String type fields support lookup table configuration. Click to enter the Configure Lookup Table dialog box. In the Configure Lookup Table dialog box, configure the related parameters. Configure lookup table: The default is not to configure. You can select Reference Tables to configure the corresponding lookup table for the metric. Lookup table source: Currently only Manual Configuration is supported. Lookup table name: Enter the name of the lookup table. It can contain Chinese characters, letters, digits, and special characters, and can be up to 128 characters in length. Lookup table description: Enter a brief description of the lookup table, up to 1,000 characters. Code information: Supports single input and batch input, with a maximum of 500 groups. Single input: Click Add Code Value, enter Code value and Code name, both of which must not be empty and must be unique. Additionally, the code value type must match the metric's value type. You can click to delete the current row. Batch input: Click Batch Input, and in the Batch Input Code Information dialog box, you can batch input code values and code names, with each group separated by a line break and code values and code names separated by a colon (:). After clicking Detect, the system will automatically parse the code information in the batch input box and fill it into the code information list. Clear all: Click Clear All, and the system will automatically clear the information list. Click OK to complete the code value configuration. Note When batch inputting code information, if there are duplicate code values or code names, after clicking OK, the system will automatically locate the first error line. Operation: Click to delete the currently configured metric. Batch delete: You can batch delete selected metrics.

   • O&M Configuration

     Note

     If the dataset update method is Manual Update, no configuration is required.

     1. Scheduling cycle

        • Planned update time: Supports scheduling at a specific time of day. This means the task runs automatically once a day, and you can specify the execution time as needed.

        • Conditional scheduling: After enabling conditional scheduling, you need to select Configuration Method, which can be either Custom Settings or Scheduling Template.

          Important

          For multiple scheduling conditions (up to 10), the system evaluates conditions in top-down order. When a condition is met, the system executes the corresponding scheduling action and terminates the evaluation of all subsequent conditions. If no condition is met, the default scheduling configuration is executed.

          Conditional scheduling only takes effect when the scheduling type is Normal scheduling.

          Scheduling conditions and trigger times are calculated using the configured scheduling time zone.

          • Custom settings

            1. Click +Add Scheduling Condition.

            2. In the Edit Conditional Scheduling dialog box, configure the conditional scheduling information.

               • Condition name: Supports any characters, up to 32 characters in length.

               • Effective status: Enabled by default. When disabled, this conditional scheduling will be ignored during scheduling.

               • Meet the following conditions: The judgment rule for the condition. When the condition evaluates to true, scheduling is performed according to the Execute scheduling configuration. For configuration instructions, see Conditional scheduling rule description.

               • Execute scheduling: Supports custom and follow scheduling attributes:

                 • Custom: When the condition evaluates to true, scheduling runs according to the configured Scheduling type.

                 • Follow scheduling attributes: Consistent with the scheduling strategy in the scheduling attributes, equivalent to the scheduling settings when conditional scheduling is disabled.

               • Scheduling type: For configuration instructions, see Scheduling type.

            3. Click OK.

               After completing the conditional scheduling settings, click Preview scheduling plan to view the dates when conditional scheduling is triggered in the calendar.

               Important

               • When you modify conditional scheduling settings and publish them to the production environment, the changes will take effect in real time for instances that are in the Not running state at the time of publishing, but not for instances that have already entered the Waiting for runtime state.

               • When using cross-node parameter judgment types in conditional scheduling, you must provide possible parameter values for preview.

          • Scheduling template

            After selecting Scheduling Template, you can select any Conditional scheduling template configured in Planning > Common business logic > Offline scheduling template. If there is no template that meets your requirements, you can click Create Scheduling Template to create a new one. After selecting a template, you cannot add scheduling conditions. You can click the View Details icon after each scheduling condition to view the conditional scheduling details.

            Note

            If the current task's scheduling cycle is daily, weekly, or monthly, the trigger time parameter in the referenced conditional scheduling template will take effect. If the current task's scheduling cycle is hourly or by minute, the trigger time parameter will not take effect.

        • Scheduling plan: Click Preview, and the scheduling plan will display all scheduling instances and their scheduling types for each day of a month according to the configured scheduling cycle and conditional scheduling. The preview date type can be selected as either Data timestamp or Run date (scheduling date).

          If there are multiple scheduling type statuses for all instances in a day, they are displayed by color showing all included scheduling type statuses, along with the name of each scheduling type status and the corresponding number of instances. For example, the figure below shows that on the 4th of a month, the current scheduling task has 44 normal scheduling instances, 2 paused instances, and 12 dry-run instances.

          Hover the mouse over a scheduling type module on a certain day to view the detailed scheduling instance list for the current scheduling task on that day, including runtime, scheduling type, and condition name.

     2. Scheduling dependency

        Scheduling dependency refers to the upstream and downstream dependency relationships between nodes. In Dataphin, downstream task nodes start running only after upstream task nodes have completed running successfully.

        • Auto parsing

          The system will automatically parse upstream dependency nodes based on task lineage and associate them. Data updates will depend on upstream data output.

          Note

          • If the auto-parsing result does not meet your expectations, you can click to turn off the effective button, which means you choose not to depend on that node.

          • The default is current cycle dependency.

        • Add dependency

          If Auto Parsing cannot parse the scheduling dependency relationship or the upstream dependency configuration generated by Auto Parsing does not match the actual application, you can manually add Upstream dependency for the node.

          Click Add Dependency, select to add Physical node or Logical table node, select one or more target physical nodes or logical table nodes in the pop-up dialog box, and then click OK.

          Note

          • If you have not purchased the Intelligent Development edition, you can only add Physical node dependencies.

          • After manually adding dependencies, if you click auto parsing again and the generated parsing nodes are the same as the manually added dependency nodes, the system will automatically overwrite them.

        • Edit dependency

          In the scheduling dependency list, click the icon in the Actions column of the target upstream dependency table. In the pop-up dialog box, you can modify these settings Dependency cycle, Dependency policy, or Dependent fields (only logical table nodes support field modification). For dependency configuration and description, see Configure offline task scheduling dependencies and Schedule dependency scene rules and examples.

          Click the Actions icon in the Actions column of the target upstream dependency table to delete the corresponding dependency node.

6. Click Save And Publish to complete the creation of the offline dataset.

   Note

   After saving successfully, you can click Preview. The system will display the corresponding data information based on the configured processing logic to help you verify whether the processing logic is correct.

What to do next

After completing the creation and configuration of the offline dataset, you can create corresponding offline tags for the offline dataset. For more information, see Offline tag.