Guidelines for submitting offline computing nodes

Updated at:

When you develop and submit an offline computing node, Dataphin automatically parses the data lineage of tables and fields in the node. The system also identifies the change type and content of the node object and runs pre-checks on the submitted node. These checks ensure that your node meets submission requirements and reduce the risk of submitting a faulty node.

Lineage Relationship Parsing Description

Data lineage parsing: When you submit a node, the system parses the data lineage in the development environment. When you publish the node, the system parses the data lineage in the production environment. The system can parse up to 100,000 data lineage records. If this limit is exceeded, the records are not saved and cannot be displayed in the Asset folder.

Submission details

When you submit an offline computing node, the Submitting Log dialog box appears. In this dialog box, you can view the submission content and pre-check results for the node, and add submission comments.

  • Submission content

    View the object name, object type, change type, and change content for the submitted node object. The change content includes Basic Information, Compute Code, Runtime Parameters, Scheduling Configuration, Scheduling Dependencies, Runtime Configuration, and .

  • Pre-checks

    When you submit an offline computing node, the system performs the following pre-checks. You cannot submit the node until all check items are configured correctly.

    Check Item

    Description

    Scheduling Dependencies

    Dataphin uses the scheduling dependency configurations of each node to run the nodes in a business flow in an orderly manner. This ensures that business data is generated efficiently and on time. For more information, see Configure scheduling dependencies for an offline node.

    Runtime Parameters

    Runtime parameter settings assign values to the variables used in the node's code to support node scheduling. Parameter variables are automatically replaced with their corresponding values. For more information, see Configure runtime parameters for an offline node.

    Cross-node Parameters

    Variable parameters that are passed through to the immediate descendant nodes of this node object. For more information, see Configure runtime parameters for an offline node.

  • Submission comments

    Add comments for this submission. Comments can be up to 128 characters long.

Check items

After all pre-checks pass, click OK and Submit. In the Submitting Log dialog box, you can view the check items and results for the node.

Check Item

Description

Configuration Check

This check includes pre-checks, code length, referenced resource count, and offline code template version.

  • Pre-checks: The system checks whether all required properties are filled in.

  • Code length: Checks whether the code is within the 100,000-character limit. If the code exceeds the limit, you cannot submit it.

  • Referenced resource count: Checks whether the number of referenced resources is within the limit of 100.

  • Offline code template version: Checks whether the offline code template version has changed.

Parameter Settings

The system parses all variables in the code and displays them in a list. The list includes the parameter name, parameter type, and current actual value. The system checks whether local variables are assigned values.

Syntax Check

For SQL nodes only. The system checks the SQL syntax for correctness. If errors exist, you cannot submit the node.

Object Check

For SQL nodes only. Checks whether all referenced objects are submitted and published to the production environment. The system parses all referenced objects in the code and displays them in a list. The list includes the object name, object type, and check result.

  • Object name: The format is [catalog].[object_name].

    • Project name variable: If the catalog in the code is the project name variable ${catalog}, the catalog is the name of the production project.

    • Hard coding: If the name of the development or production project is hard-coded, the original name is used. If you omit the catalog, the system uses ${catalog} by default.

    • Omitting the catalog: If you omit the catalog, the system uses the ${catalog} project name variable by default.

  • Object type: The checked object types include physical tables, fields, and logical tables.

  • Check result: Includes Success and Failed.

    • Success: All checked objects are submitted and published to the production environment.

      Note

      If the catalog is hard-coded, the system only checks whether the object exists in the corresponding environment.

    • Failed: One or more checked objects are not submitted or not published to the production environment.

      Note

      The check also fails if a field in a checked object does not exist.

Permission Check

The system parses all objects in the code and displays a list indicating whether you have the required operation permissions for the objects. The list includes the object name, object type, check result, row-level permission status, and permission request operation.

  • Object name: The format is [catalog].[object_name].

    • Project name variable: If the catalog in the code is the project name variable ${catalog}, the catalog is the name of the production project.

    • Hard coding: If the name of the development or production project is hard-coded, the original name is used. If you omit the catalog, the system uses ${catalog} by default.

    • Omitting the catalog: If you omit the catalog, the system uses the ${catalog} project name variable by default.

  • Object type: The checked object types include physical tables, global objects, logical tables, and functions.

    Note

    If authentication for cross-project references is enabled for the project where the referenced function resides, authentication is required for both the personal account and the tenant account during pre-editing, running, and submission.

  • Check result: Checks whether the current user and the tenant account have read and write permissions for the object type. The result can be Success or Failed.

    Note
    • If the object is a table, you must have read and write permissions for all referenced fields.

    • If the object is a function, the tenant account must have the required read and write permissions for the function in a Basic project. In a Dev-Prod project, the personal account must have the required read and write permissions.

  • Row-level permission status: This feature requires the purchase of the row-level permissions value-added service. If a table has row-level permission control enabled and you do not have the required permissions, the system automatically adds the Row-level permission status column. This column notifies accounts that lack row-level permissions. The status can be Failed, Success, or Success with threat.

    • Failed: The status is Failed if the personal account does not have row-level permissions for the corresponding object.

    • Success with threat: If the project is a Dev-Prod project, the status is Success with threat if the personal account has row-level permissions but the tenant account does not.

  • Permission request: If a check on an object fails, you can click the image.png icon at the bottom to request permissions. For more information, see My Permissions.

Standard Check

For SQL nodes only. The system scans the SQL code based on built-in standards and displays the check result for each standard. For more information, see Coding standards.

Dependency Check

Checks the dependency configuration and output configuration.

  • Dependency configuration: Checks whether the dependent node objects configured for the node exist in the development environment.

  • Output configuration: Checks whether the output name of the node is unique within the current tenant.

Code Review

The system automatically checks whether the node has configurations that trigger a code review. If so, a code review is required. For example, a code review is triggered if the node references a global variable for which code review is enabled. For more information, see View and configure built-in approval templates.

Important
  • If code review is enabled for the project where the current computing node resides, a code review is performed upon submission, regardless of whether the offline computing node references a global variable that requires a review.

  • If code review is enabled for the project and the computing node references a global variable that requires a code review, all approval tasks in the code review check item must be passed before you can submit the node.

Operation Execution

When this check is reached, the system executes the submission. You cannot revoke the submission while the operation is in progress.