Limits

Feature

Operation

Limits/Recommendations

Management Center

Member management

• The super administrator is a role that the system automatically generates during initialization after you purchase Dataphin.

  • Your Alibaba Cloud account is the super administrator account. You can create Resource Access Management (RAM) users with the Resource Access Management feature. For more information, see Prepare a RAM user.

  • A Dataphin system has only one super administrator account, which has all permissions in the system.

• To update the user list or user information for RAM users, perform the following steps:

  1. Log on to the Management Console with the super administrator account and configure an AccessKey for authorization.

  2. Use the super administrator account to perform a Sync Account System operation. This retrieves the RAM users under the Alibaba Cloud account and adds them as Dataphin members.

Management Center

Configure compute engine

• Global configurations can be updated only by the super administrator when no compute engine source exists in the system.

• For MaxCompute compute engines, see Endpoint for details on how to configure the Endpoint address.

Compute engine type

Select settings

Purchase MaxCompute resources before you configure the compute engine. The system requires these resources to support data development.

• Select a compute engine type. Currently, only MaxCompute is available. Configure the cluster where the compute engine resides, including information such as the Endpoint. The system uses this configuration to support data development on the cluster. Configure the settings based on your compute engine cluster.

• To use Spark on MaxCompute for computation, submit a ticket to confirm if the Spark service is active in the region of your MaxCompute compute engine. If the Spark service is not active in the region, your Spark nodes cannot run successfully.

• Use Dataphin as the only entry point for data development and management. This prevents issues such as metadata faults and abnormal permissions.

Data Source Management

Add data source

• Configure the data source AccessKey with administrative permissions. You can do this by configuring the AccessKey of your Alibaba Cloud account or by granting all MaxCompute permissions to the AccessKey of a RAM user.

• Do not configure the same physical database as two different data sources if the configurations are identical.

Project management

Project name

• When you set the data source type to MaxCompute, the project's English name must match the MaxCompute project's English name.

• Project names cannot start with LD_ or ld_. This prevents conflicts with business segment names, which can cause the query feature to become unavailable.

Project management

Configure compute engine source

• For a physical database that is configured as a project data source, do not add, delete, or modify its data from any console other than Dataphin.

• Do not configure cross-cluster compute engine sources for a project.

R&D Workbench

Standard modeling

• Name standard definitions and logical table objects carefully. Use lowercase English letters for the names. This prevents names from becoming unchangeable or hard to read due to downstream dependency constraints.

• Use English abbreviations whenever possible for field names. This prevents names from exceeding database limits and causing errors during data generation.

R&D Workbench

Data processing

• You cannot read data if the project's compute engine source spans multiple clusters.

• For tables not created in Dataphin, Dataphin might not be able to retrieve or update their metadata.

R&D Workbench

Ad hoc query

When you query a logical table, you must use the English name of the business segment as a prefix. When you use a physical table from another project, you must use the English name of that project as a prefix.

To query data in the development environment, append _dev to the production name. The system automatically generates corresponding variables for production business segments and projects. For example, if you have a business segment named LD_Trade, the system generates the variable ${LD_Trade}. When run in the development environment, this variable is replaced with LD_Trade_dev by default. When run in the production environment, it is replaced with LD_Trade. You can also set a fixed value at runtime to make your code more flexible across different environments.