Before creating a project space for data development, you must configure a compute engine for your Dataphin instance. After the compute engine is configured, you can add the corresponding compute source to a project space to provide compute and storage resources.
Permissions
Only a super administrator or a system administrator can configure compute engines.
Billing
To configure a real-time compute engine, you must first purchase and enable the Real-time R&D module.
The Agile R&D Edition does not support real-time compute engines.
Limitations
-
After configuring a compute engine for a business tenant, reconfiguring the metastore compute engine type may cause incorrect metadata processing for that tenant. We recommend that you contact the Dataphin operations team for confirmation before you modify the metastore compute engine type.
-
The default real-time compute engine is Realtime Compute for Apache Flink. To use Blink, go to the Management Center to change the real-time compute engine type.
-
When you modify the offline compute engine settings, the system automatically updates the compute source configuration. To ensure efficiency, the system does not verify the connectivity of the compute source during this process. Ensure the configuration is accurate to prevent task failures. After the modification is complete, we recommend that you manually test the connectivity of the compute source.
-
After you modify the compute settings, the new configuration takes effect on the compute source within 30 seconds. Before the synchronization is complete, you may see inconsistencies when viewing the compute source configuration, and SQL execution may still use the previous settings.
Supported compute engines
In single-engine mode, configure a compute engine for your Dataphin instance by specifying its cluster address. Once configured, you can create compute sources based on this cluster. Dataphin supports the following compute engines:
If no offline compute source exists, you can change both the compute engine type and its configuration. If one already exists, you can only modify the configuration, not the engine type.
If the tenant's metastore compute engine is already initialized, you can only select compute engines supported by that metastore.
|
Compute engine |
Description |
References |
|
Offline compute engine |
||
|
MaxCompute |
An Alibaba Cloud-native big data computing platform that provides efficient, stable storage and computing for massive datasets. |
Set MaxCompute as the compute engine for a Dataphin instance |
|
Real-time compute engine |
||
|
Realtime Compute for Apache Flink |
An Alibaba Cloud service based on Apache Flink that supports both real-time and offline batch processing with high throughput and low latency. |
After you enable the Real-time R&D module for a tenant, the system recommends a configuration based on the selected offline compute engine. You can modify this setting as needed. |
|
Blink exclusive |
Alibaba Cloud's real-time compute engine. Important
This version is no longer sold on the public cloud. Use with caution. |
|