Offline computing task submission instructions-Dataphin(Dataphin)-阿里云帮助中心

When you submit an offline computing task, Dataphin automatically parses the lineage for tables and fields, identifies the type and content of any changes, and runs a series of pre-checks. These validations help ensure your task meets all submission requirements and reduce the risk of errors.

Lineage parsing

Dataphin parses lineage in the development environment when you submit a task, and in the production environment when you publish it. The system records up to 100,000 lineage relationships. If this limit is exceeded, the lineage is not recorded and cannot be displayed in the asset catalog.

Commit details

When you submit an offline computing task, the Commit Details dialog box lets you view the submission content, pre-check results, and add commit notes.

Submission content
You can view the object name, object type, change type, and change content for the submitted task. The change content includes Basic Information, Code, Runtime Parameters, Scheduling Configuration, Scheduling Dependencies, Execution Configuration, and Resource Configuration.

Pre-check

When you submit an offline computing task, the system performs the following pre-checks. You cannot submit the task until all check items are configured correctly.

Check item	Description
Check Conditions	Checks whether the check conditions for the node are configured. For more information, see Check conditions. Note This check item applies only to check node tasks.
Scheduling dependency	Dataphin uses scheduling dependencies to run business process nodes in the correct order, ensuring efficient and timely data generation. For more information, see Configure scheduling dependencies for an offline task.
Runtime parameter	These parameters enable node scheduling, and the system automatically replaces them with their corresponding values during execution. For more information, see Configure and use node parameters.
Cross-node parameter	Variable parameters passed to the immediate downstream nodes of the current object. For more information, see Configure and use node parameters.
Dataset lineage configuration	For Python/Shell tasks, if the referenced dataset is inconsistent with the lineage, the submission fails. For more information, see Configure datasets referenced by Python/Shell tasks and Custom lineage configuration.

Commit notes
You can add notes for this commit, up to 128 characters long.

Check items

After all pre-checks pass, click OK and Submit. In the Commit Details dialog box, you can view the check items and their results.

Check item	Description
Configuration check	This includes the following checks: pre-check, code length, number of referenced resources, and offline code template version. Pre-check: The system verifies that all required properties are filled in. Code length: Checks if the code length is within the 100,000 character limit. Submissions exceeding this limit will fail. Number of referenced resources: Checks if the number of referenced resources is within the limit of 100. Offline code template version: Checks for any changes in the offline code template version.
Parameter Configuration	The system parses all variables in the code and displays them in a list, showing the parameter name, parameter type, and current value. It also checks whether local variables have been assigned a value.
Syntax check	For SQL tasks only. The system validates the SQL syntax. The task cannot be submitted if errors are found.
Object check	For SQL tasks only. The system checks whether all referenced objects have been committed and published to the production environment. It parses all referenced objects in the code and displays them in a list, including the object name, object type, and check result. Object name: The format is `[catalog].[object_name]`. Project name variable: If `catalog` in the code is the project name variable `${catalog}`, then `catalog` is the production project name. Hard-coded value: If the development or production project name is hard-coded, the original name is used. If the catalog is omitted, the system defaults to using `${catalog}`. Omitted catalog: If the catalog is omitted, the system defaults to using the `${catalog}` project variable. Object type: Object types checked include physical tables, fields, and logical tables. Check result: The result can be Succeeded or Failed. Succeeded: All checked objects have been committed and published to the production environment. Note If the catalog is hard-coded, the system only checks whether the object exists in the corresponding environment. Failed: One or more checked objects have not been committed or published to the production environment. Note The check also fails if a referenced field does not exist.
Permission check	The system parses all objects in the code and displays a list indicating whether you have the required permissions for each object. This list includes the object name, object type, check result, row-level security status, and an option to apply for permissions. Object name: The format is `[catalog].[object_name]`. Project name variable: If `catalog` in the code is the variable `${catalog}`, then `catalog` is the name of the production project. Hard-coded value: If the development or production project name is hard-coded, the original name is used. If the catalog is omitted, the system defaults to using `${catalog}`. Omitted catalog: If the catalog is omitted, the system defaults to using the `${catalog}` project variable. Object type: Object types checked include physical tables, global objects, logical tables, and functions. Note When a referenced function is in a project where cross-project referencing requires authentication, both your personal account and the production account must be authenticated during development, execution, and submission. Check result: Checks whether the current user and the production account have read and write permissions for the object. The result can be Succeeded or Failed. Note If the object is a table, read and write permissions are required for all referenced fields. If the object is a function in a Basic project, the production account requires the corresponding function permissions. In a Dev-Prod project, your personal account requires the function permissions. Row-level security status: This feature requires a paid add-on for row-level security. When row-level security is enabled for a table, the system adds a Row-level security status column to alert you if an account lacks the required permissions. The status can be Succeeded, Failed, or Succeeded with Risks. Failed: The status is Failed if your personal account lacks the required row-level security permissions for the object. Succeeded with Risks: In a Dev-Prod project, the status is Succeeded with Risks if your personal account has row-level security permissions, but the production account does not. Apply for Permissions: If an object check fails, you can click the icon at the bottom to apply for permissions. For detailed steps, see My Permissions.
Specification check	For SQL tasks only. The system scans the SQL code against built-in specifications and displays the results for each rule. For more information, see Coding specifications.
Dependency check	Verifies the dependency configuration and output configuration. Dependency configuration: Checks whether the dependent node objects configured for the task exist in the development environment. Output configuration: Checks whether the task's output name is already used within the current tenant.
Code review	The system automatically checks for task configurations that trigger a code review. If found, a review is required. For example, this can happen if the task references a global variable that requires code review. For more information, see View built-in approval templates. Important If code review is enabled for the task's project, a code review is required upon submission, regardless of whether the task references global variables that require a review. If code review is enabled for the project and the task references global variables that require a review, all approval tasks within the code review check must pass before you can submit the task.
Operation execution	When the check reaches this item, the system proceeds with the task submission. You cannot cancel the submission during this process.