Bind an EMR Serverless Ray compute resource
To run Ray jobs in DataWorks, bind a Ray cluster from an EMR Serverless Spark workspace as a Serverless Ray compute resource. You can then select this resource on a Serverless Ray node in Data Studio to run tasks.
Prerequisites
-
An EMR Serverless Spark workspace is created in E-MapReduce, with an available Ray cluster in the workspace.
-
A DataWorks workspace is created, your RAM account is added to the workspace, and the workspace administrator role is assigned to the account.
ImportantOnly workspaces that Use Data Studio (New Version) are supported.
-
A serverless resource group is created and bound to the target DataWorks workspace.
Limitations
-
Region restrictions: The supported regions are the same as those for binding an EMR Serverless Spark compute resource, including China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), and US (Virginia). The regions displayed in the console prevail.
-
Permission requirements:
Operator
Permissions
Alibaba Cloud account
No additional permissions are required.
RAM account/RAM role
-
DataWorks administrative permissions: Only workspace members with the O&M and workspace administrator roles, or the
AliyunDataWorksFullAccesspermission, can create compute resources. For details, see Grant a user the workspace administrator permission. -
EMR Serverless Spark service permissions: You must have the
AliyunEMRServerlessSparkFullAccesspermission policy and theOwnerpermission for the target Spark workspace. For details, see Manage users and roles.
-
Bind a Serverless Ray compute resource
On the Compute Resources page, bind a Ray cluster as a Serverless Ray compute resource.
-
Select the type of compute resource to bind.
-
Click Associate Computing Resources to go to the Associate Computing Resources page.
-
On the Associate Computing Resources page, select Serverless Ray as the compute resource type to open the Bind Serverless RAY Compute Resource wizard.
-
-
Fill in the binding information in the wizard.
On the Enter Information page, configure the following parameters.
Parameter
Description
Spark Workspace
Select the EMR Serverless Spark workspace that contains the Ray cluster. You can also create a Spark workspace from the drop-down list.
Billing Method
Inherited from the selected Spark workspace, such as Pay-as-you-go. This value cannot be modified.
RAY Cluster
Select the Ray cluster to bind. The list shows all Ray clusters in the selected Spark workspace.
Engine Version
Auto-populated based on the selected Ray cluster, showing the engine version and built-in Ray and Python versions. The versions displayed in the console prevail.
Computing Resource Instance Name
A unique name that identifies this compute resource in DataWorks. When you run a task, select this instance name on the node to use this binding.
Description
Optional. A brief note about the business purpose of this compute resource.
ImportantTo properly retrieve cluster information in DataWorks, do not remove the administrator role of the DataWorks service-linked roles
AliyunServiceRoleForDataWorksOnEmrandAliyunServiceRoleForDataWorksEnginefrom the E-MapReduce Serverless Spark workspace. -
Click Confirm to complete the Serverless Ray compute resource binding.
Next step
After binding, you can create a Serverless Ray node in Data Studio and select this compute resource to run Ray jobs.