To develop and manage Hologres tasks in DataWorks, you must first associate your Hologres instance as a DataWorks computing resource. After the association is complete, you can use the computing resource in various DataWorks modules for data synchronization, data development, and data analysis.
Prerequisites
-
You have purchased a Hologres instance and created a database.
Important-
When you purchase the Hologres instance, ensure that it is in the same region as your DataWorks workspace. Otherwise, the association will fail.
-
Before you associate the instance, make sure that its Running status is normal.
-
-
You have created a DataWorks workspace. The RAM user performing the operation has been added to the workspace and assigned the Workspace Administrator role.
-
You have associated a resource group with the workspace and ensured network connectivity.
-
If you use a Serverless resource group, ensure that it can connect to the Hologres computing resource.
-
If you use a legacy exclusive resource group, ensure that the corresponding exclusive resource group for integration, exclusive resource group for scheduling, and exclusive service resource group can connect to the Hologres computing resource.
-
Limitations
-
Feature limitation: If you enable SSL authentication when you create a computing resource, the resource cannot be used for Data Development or periodic scheduling tasks.
-
Region limitation: China (Hangzhou), China (Shanghai), China (Beijing), China (Ulanqab), China (Shenzhen), Japan (Tokyo), Malaysia (Kuala Lumpur), and Indonesia (Jakarta).
-
Permission limitation:
Operator
Required permissions
Alibaba Cloud account
No additional permissions are required.
RAM user or RAM role
DataWorks management permissions: Only workspace members who are assigned the O&M or Workspace Administrator role, or who have the
AliyunDataWorksFullAccesspolicy can create computing resources. For more information, see Grant the Workspace Administrator permission to a user.Hologres service permissions:
-
You must grant the
AliyunHologresFullAccessaccess policy. Otherwise, you cannot configure the Hologres computing resource. -
You must add the RAM user or RAM role to User Management to select the destination database during the configuration.
-
Data Studio (new version): Associate a Hologres computing resource
This section describes how to associate a Hologres computing resource in a workspace that Use Data Studio (New Version).
Go to the computing resources page
-
Log on to the DataWorks console. After you switch to the target region, click in the left-side navigation pane. In the drop-down list, select the target workspace and click Go to Management Center.
-
In the left-side navigation pane, click Computing Resources to open the computing resources page.
Associate a Hologres computing resource
On the computing resources page, configure and associate a Hologres computing resource.
-
Select the type of computing resource to associate.
-
Click Associate Computing Resources. The Associate Computing Resources page appears.
-
On the Associate Computing Resources page, select Hologres as the computing resource type. The Associate Hologres Computing Resource configuration page appears.
-
-
Configure the Hologres computing resource.
On the Associate Hologres Computing Resource configuration page, configure the parameters as described in the following table.
Parameter
Description
Hologres instance
Select the Hologres instance. You can also click Create in the drop-down list to purchase a Hologres instance.
Hologres Virtual Warehouse
Select a compute group for the instance. You can also click Create in the drop-down list to create a Hologres compute group.
Database name
Select the database. You can also click Create in the drop-down list to create a database.
Default access identity
-
Development environment: Only the Executor identity is supported.
-
Production environment: You can use an Alibaba Cloud Account, a Alibaba Cloud RAM Sub-account, a Alibaba Cloud RAM role, or the Task Owner identity.
Note-
If you are logged on with an Alibaba Cloud account: You can select all identities.
-
If you are logged on with a RAM user or RAM role:
-
If you have the AdministratorAccess policy: You can select all identities.
-
If you do not have the AdministratorAccess policy: You can select only your own identity.
-
-
Authentication option
-
No Authentication: No other operations are required.
-
SSL Authentication: If you select this authentication method, you must enable the SSL Encryption switch in Data Security of the Hologres instance.
NoteIf SSL authentication is enabled for a Hologres instance, it cannot be used for Data Development or periodic scheduling tasks.
SSL encryption mode
This parameter is required when Authentication Method is set to SSL Authentication. Only the require encryption mode is supported.
Computing resource instance name
Specify a custom name for the computing resource instance. When a task is running, you can select the computing resource for the task based on the computing resource name.
-
-
Test the connectivity.
In the connection configuration section, select the resource group that DataWorks uses to run Hologres tasks and click Test Connectivity to ensure that the resource group can access your Hologres instance. For more information, see Network connectivity solutions.
-
Click Confirm to complete the Hologres computing resource configuration.
NoteAfter the binding is complete, the system automatically creates a Hologres data source with the same name in the Data Sources of the current workspace.
Data Studio (legacy): Associate a Hologres computing resource
Associate a Hologres computing resource with a workspace that does not Use Data Studio (New Version).
Go to the computing resources page
Go to the DataStudio page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Development.
-
In the left-side navigation pane, click the
icon to go to the Computing Resources page.
Associate a Hologres computing resource
On the computing resources page, configure and associate a Hologres computing resource.
-
Select the type of computing resource to associate.
-
Click Create Computing Resource to go to the Create Computing Resource page.
-
On the Create Computing Resource page, select Hologres as the computing resource type to go to the Create Computing Resource configuration page.
-
-
Configure the Hologres computing resource.
On the Create Computing Resource configuration page, configure the parameters as described in the following table.
Parameter
Description
Data source name
Specify a custom name for the computing resource.
Authentication method
Only Alibaba Cloud Account and Alibaba Cloud RAM Role is supported.
Cloud account
You can select Current Alibaba Cloud Account or Another Alibaba Cloud Account.
NoteIf you use Another Alibaba Cloud Account, you can enter the parameter information of the other account as prompted on the page.
Region
The region where your workspace resides.
Hologres instance
Select the Hologres instance that you created.
Database name
Select the database that you want to use.
Default access identity
-
Development environment: Only the Executor identity is supported.
-
Production environment: You can use an Alibaba Cloud Account, a Alibaba Cloud RAM Sub-account, a Alibaba Cloud RAM role, or the Task Owner identity.
Note-
If you are logged on with an Alibaba Cloud account: You can select all identities.
-
If you are logged on with a RAM user or RAM role:
-
If you have the AdministratorAccess policy: You can select all identities.
-
If you do not have the AdministratorAccess policy: You can select only your own identity.
-
-
Authentication option
-
No Authentication: No other operations are required.
-
SSL Authentication: If you select this authentication method, you must enable the SSL Encryption switch in Data Security of the Hologres instance.
NoteIf SSL authentication is enabled for a Hologres instance, it cannot be used for Data Development or periodic scheduling tasks.
SSL encryption mode
This parameter is required when Authentication Method is set to SSL Authentication. Only the require encryption mode is supported.
-
-
Test the connectivity.
In the connection configuration section, select the resource group that DataWorks uses to run Hologres tasks and click Test Connectivity to ensure that the resource group can access your Hologres cluster. For more information, see Network connectivity solutions.
-
Click Create and Associate Computing Resource with DataStudio to complete the Hologres computing resource configuration.
After the binding is complete, the system automatically creates a Hologres data source with the same name in the Data Sources of the current workspace.
Next steps
To ensure a smoother development process, we recommend that you read Usage of DataWorks on Hologres to understand the development process, related fees, environment preparation, and permission control for using Hologres in DataWorks.
-
Data Studio (new version): After configuring the Hologres computing resource, you can perform data synchronization operations through batch synchronization nodes and data development operations through Hologres-related nodes in Data Development.
-
Data Studio (legacy): After configuring the Hologres computing resource, you can perform data synchronization operations through nodes and data development through legacy Hologres-related nodes in Data Development.
FAQ
-
Error message: FAILED: Build connection error! Connection to xxx.hologres.aliyuncs.com:80 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
Solution: Go to the Hologres management console to check if the instance is running properly. If the instance is down, you can restart it.
-
Error message: FAILED: An I/O error occurred while sending to the backend.
Solution: In Data Security of the Hologres instance, check if the SSL Encryption switch is enabled. If it is enabled, you can disable it and run the task again after 5 to 10 seconds.