Submitting real-time computing tasks-Dataphin(Dataphin)-阿里云帮助中心

After you develop and submit a real-time computing task, Dataphin automatically parses the data lineage for its tables and fields. Dataphin also confirms the change type and content of the task object and runs pre-checks on your submission. These checks ensure that the task meets all submission requirements and reduce the chance of submission errors.

Data lineage parsing

Dataphin parses the data lineage in the development environment during task submission and in the production environment during publication. A maximum of 100,000 data lineage records can be parsed. Records that exceed this limit are not saved and cannot be displayed in the asset directory.

Submission details

When you submit a real-time computing task, the Submit dialog box opens. In this dialog box, you can view the submission content, pre-check results, and add submission notes.

Submission content
This section displays the object type, task pattern, change type, and change details of the submitted task object. The change details include Basic Information and Task Details.

Pre-check

When you submit a real-time computing task, Dataphin performs the following pre-checks. You cannot submit the task if any check item is configured incorrectly.

Check item	Description
Real-time pattern
Variable configuration	Variable configuration involves checking the variables used in the computing task's code. This includes checking if local variables are set and if parameter names are valid. For more information, see Configure variables for the real-time pattern.
Offline pattern
Scheduling dependencies	Dataphin uses the scheduling dependency configurations of each node to run the nodes in a business flow in an orderly manner. This ensures that business data is generated efficiently and on time. For more information, see Configure scheduling for the offline pattern.
Variable configuration	Variable configuration involves checking the variables used in the computing task's code. This includes checking if local variables are set and if parameter names are valid. For more information, see Configure variables for the offline pattern.

Submission notes
You can add notes for the task submission. The notes must be 128 characters or less.

Check items

After you submit a real-time computing task, you can view the check items and their results in the Submit dialog box.

Check item	Description
Variable configuration	The system parses all variables in the code and displays them in a list. It checks if parameter names are valid (only letters, numbers, and underscores are supported) and if local variables have been assigned a value.
Syntax check	Flink_SQL tasks undergo a syntax check to verify SQL syntax. If a syntax error is detected in a Flink_SQL task, click Locate Error to navigate to the line with the error in the editor. Note If Dataphin cannot locate a syntax error in the code, it displays an error message.
Object check	Flink_SQL tasks undergo an object check to verify that all referenced objects are submitted and published to the production environment. The system parses all referenced objects in the code and displays them in a list that includes the object name, object type, and check result. Object type: The checked object types include meta tables, image tables, physical tables, data source tables, functions, and resources. Check result: Includes Success and Failure. Success: All checked objects are submitted and published to the production environment. Note If the catalog is hard-coded, the system only checks whether the object exists in the corresponding environment. Failure: One or more checked objects have not been submitted or published to the production environment. Note The check also fails if a field does not exist in a checked object.
Compilation check	Checks whether tables and fields exist in the code, whether functions are used correctly, and whether field types are supported.
Permission verification	The system parses all objects in the code and lists whether the current user has the required operation permissions for them. If the permission verification is successful, the user has the necessary permissions for the checked objects. If the permission verification fails, the user does not have the required permissions. A list of objects for which permissions are missing is displayed in the permission verification dialog box. Object name: The name of the object for which permission is missing. The name format differs based on the object type. Object type: Tables and global variables. Permission type: The permissions required to run the script. Data source tables require only table data read permission. Data sources require only execute permission. Operation: If an object check fails, you can request permissions for one or more objects. To request personal permissions for a single object: Click the icon in the Actions column of the target object. The default validity period is 30 days. To request permissions for multiple objects, select the objects, or select Select All Tables, and then click Request Permissions. For more information about how to request permissions, see Request permissions. After you obtain the permissions, you can run the task again.
Dependency check	Checks the dependency configuration and output configuration. Dependency configuration: Checks whether the dependency node objects configured for the task exist in the development environment. Output configuration: Checks whether the output name of the task is already used in the current tenant.
Code review	If a task contains content that triggers a code review, the system performs a code review check. For example, a check is triggered if the task references a global variable for which code review is enabled. The code review check automatically generates a ticket and submits it to the reviewer. You can perform Revoke and Open Ticket operations on the code review request.