Job configuration description
This topic describes the common configuration items and configuration methods for RAY jobs on the Lindorm compute engine.
Configuration overview
Ray jobs support two categories of configurations: job-level and cluster-level:
-
jobConfig: Job-level configurations, such as the target resource group, RuntimeEnv, and TTL.
-
clusterConfig: Cluster-level configurations, such as the image, Head/Worker resource specifications, instance type preferences, and environment variables.
jobConfig items
|
Configuration item |
Required |
Default value |
Description |
|
|
Yes |
None |
Name of the target RAY resource group. Compatible with the unified |
|
|
No |
None |
A Ray RuntimeEnv JSON string that specifies dependency packages, the working directory, and environment variables. |
|
|
No |
300 |
The retention time of the Ray cluster after the job completes, in seconds. |
clusterConfig items
Cluster-level configurations
|
Configuration item |
Required |
Default value |
Description |
|
clusterConfig.image |
No |
Platform default image |
The image address used to run the Ray job. If not specified, the default image is used. For information about how to use a custom image, see Run Ray jobs with a custom image. |
|
clusterConfig.env.<key> |
No |
None |
Cluster-level environment variables. |
Head node configuration
|
Configuration item |
Required |
Default value |
Description |
|
clusterConfig.head.cpu |
No |
4 |
Number of CPU cores for the Head node. |
|
clusterConfig.head.memoryGB |
No |
16 |
Memory of the Head node (GB). |
|
clusterConfig.head.diskSizeGB |
No |
30 |
Disk of the Head node (GB). |
|
clusterConfig.head.machineType |
No |
None |
Instance type preference for the Head node. |
|
clusterConfig.head.env.<key> |
No |
None |
Head node environment variables. |
Worker Group configuration
Worker Groups are distinguished by name. For example, to define a Worker Group named wg1, use clusterConfig.workerGroup.wg1.*.
|
Configuration item |
Required |
Default value |
Description |
|
clusterConfig.workerGroup.wg1.cpu |
No |
4 |
Number of CPU cores for the Worker. |
|
clusterConfig.workerGroup.wg1.memoryGB |
No |
16 |
Worker memory (GB). |
|
clusterConfig.workerGroup.wg1.diskSizeGB |
No |
30 |
Worker disk (GB). |
|
clusterConfig.workerGroup.wg1.replicas |
No |
2 |
Number of Worker replicas. |
|
clusterConfig.workerGroup.wg1.minReplicas |
No |
None |
Minimum number of Worker replicas. Takes effect when auto scaling is enabled. |
|
clusterConfig.workerGroup.wg1.maxReplicas |
No |
None |
Maximum number of Worker replicas. Takes effect when auto scaling is enabled. |
|
clusterConfig.workerGroup.wg1.machineType |
No |
None |
Instance type preference for the Worker. |
|
clusterConfig.workerGroup.wg1.env.<key> |
No |
None |
Worker node environment variables. |
Configuration method
Submit a Ray job by using POST /api/v1/lindorm/jobs/{token}. All configurations are passed in through the conf field of the request body.
{
"owner": "data-team",
"mainResourceKind": "ray",
"mainResource": "python my_script.py --arg1 val1",
"name": "my-ray-job",
"conf": {
"jobConfig.computeGroup": "raycgserverless",
"jobConfig.runtimeEnvJson": "{\"pip\":[\"pandas\"],\"working_dir\":\"https://your-bucket.oss-cn-hangzhou.aliyuncs.com/ray-project.zip\"}",
"clusterConfig.head.cpu": "4",
"clusterConfig.head.memoryGB": "16",
"clusterConfig.workerGroup.wg1.cpu": "8",
"clusterConfig.workerGroup.wg1.memoryGB": "32",
"clusterConfig.workerGroup.wg1.minReplicas": "1",
"clusterConfig.workerGroup.wg1.maxReplicas": "4"
}
}