Submit a job in a serverless cluster

Background information

After you submit a job in a serverless cluster, the system automatically creates elastic container instances to run the job. An elastic container instance requires the following configurations:

• Specification: You can specify the vCPU and memory for elastic container instances. You can also specify ECS instances to meet additional requirements such as GPU or enhanced network capabilities. The billing methods vary based on your specifications. For more information, see Elastic container instances.

• Container:: An elastic container instance in a serverless cluster comes with one container. Before you deploy the container, you must package the required environment and data into a container image and upload the image to Alibaba Cloud Container Registry (ACR).

• Network: An elastic container instance occupies one Elastic Network Interface (ENI) of the vSwitch in the VPC to which the elastic container instance belongs. By default, the elastic container instance has an internal IP address. If you want the elastic container instance to connect to the Internet, you must associate a NAT gateway with the VPC.

• Storage: By default, 30 GiB of temporary storage is provided free of charge for each elastic container instance. You can add more temporary storage as needed. If you need persistent storage, you can mount a NAS file system or Object Storage Service (OSS) buckets.

Prerequisites

• A serverless cluster is created. For more information, see Create a serverless cluster.

• A container image is prepared and uploaded to ACR.

  Instances of Container Registry Personal Edition and Enterprise Edition are supported. For more information, see the following topics:

  • Use a Container Registry Personal Edition instance to push and pull images

  • Use a Container Registry Enterprise Edition instance to push and pull images

• If you want to persist data, you can mount OSS buckets and NAS file systems.

  • If you use NAS, you must create a NAS file system and a mount target. For more information, see Create a file system and Manage mount targets.

  • If you use OSS, you must create an OSS bucket. For more information, see Create buckets.

Procedure

1. Go to the Operation page.

   1. Log on to the E-HPC console.

   2. In the upper-left corner of the top navigation bar, select a region.

   3. In the left-side navigation pane, choose Job and Performance Management > Operation.

2. On the Operation page, select your serverless cluster from the Cluster drop-down list.

3. Click the Submit Job tab.

4. Configure the job parameters, and then click Submit Job.

   When you submit a job, you must configure parameters related to the elastic container instances and the job. The system automatically creates elastic container instances to run the job based on the configurations.

   Note

   To submit a new job with a similar configuration, click Export to save the current settings to a local file. The next time you submit a job, you can click Import to quickly load the saved configuration.

   Parameter	Description
   Job Name	The name of the job.
   vSwitch	The vSwitch with which you want to associate the elastic container instance.
   Image URL	The URL of the container image that is uploaded to ACR. The URL is used to deploy the container.
   Enable Job Array	Specifies whether to enable the job array feature of the scheduler. The job array feature is used to submit and manage similar jobs in batch. After you enable the job array feature, you must configure the minimum length, maximum length, and step size values of the job array. The minimum length is the first index, the maximum value is the last index, and the step size value is the interval. The default value is 1. If the minimum length is 2, the maximum value is 7, and the step size value is 2, the generated job array contains three sub-jobs that are numbered 2, 4, and 6.
   Priority	The priority of the job. Valid values: 0 to 9. A larger value indicates a higher priority.
   Temporary Storage	The temporary storage space added to the container. Unit: GiB. By default, each elastic container instance provides 30 GiB of free storage quota. If you need to store data that is greater than 30 GiB, you can use this parameter to increase the temporary storage space. The added storage is billed based on the storage space. For more information, see Billing for temporary storage space.
   Timeout	The validity period of the job. After the validity period expires, the job is forcibly terminated. Unit: seconds.
   Bid Strategy	Specifies whether to create an elastic container instance of the preemptible instance type. Not preemptible instances: The instances are created as pay-as-you-go instances. This is the default value. Preemptible instance with a maximum bid price: The instances are created as preemptible instances that have a specified maximum price per hour. Auto bidding until meeting the pay-as-you-go price: The instances are created as preemptible instances for which the market price is automatically used as the bid price. The market price can be as high as the pay-as-you-go price. For more information, see Create a preemptible instance.
   CPU	The vCPUs and memory of the elastic container instances. If you do not specify the parameters, the system creates elastic container instances that have 2 vCPUs and 4 GiB of memory. For more information, see Create an instance by specifying vCPUs and memory.
   Memory
   GPU	If you want to create elastic container instances of the GPU type, you must configure the parameter to specify the number of GPUs used in the container.
   Working Path	The working directory of the container. The system runs commands in the specified directory.
   Instance Type	Creates elastic container instances by using the specified ECS instance type. For more information, see Specify ECS instance types to create an elastic container instance.
   Startup Command	The startup command of the container. The command must meet the following requirements: Single command without arguments: Enter the command directly. For example, ls. Single command with arguments: Separate the command and its arguments with a comma. For example, ls,l. Multiple commands in sequence: You must use a shell. Separate the command and its arguments with commas, and separate individual commands with a semicolon. For example, /bin/sh,-c,ls -l;hostname.
   RAM Role	The RAM role that is associated with Elastic Container Instance. For more information, see Use an instance RAM role using API operations.
   Variables	The environment variables of the container.
   Mount Volume	The data volume that is mounted to the container. OSS and NAS are supported. OSS Volume Mount Path: the directory that is mounted to the container. OSS bucket name: The name of the OSS bucket. OSS Endpoint: the endpoint of OSS. If the bucket and cluster reside in the same region, specify the internal endpoint. If the bucket and cluster reside in different regions, specify the public endpoint. OSS Path: the OSS directory that you want to mount. RAM role: The RAM role to use for authorization. When you create the RAM role, set the trusted entity to Alibaba Cloud Service, the role type to Normal Service Role, and the trusted service to Cloud Server. Then, grant the AliyunOSSFullAccess policy to the role. NAS Volume Mount Path: the directory that is mounted to the container. NAS Mount Target: the mount target of the NAS file system. NAS Path: the NAS directory that you want to mount. Mount options: The mount options. We recommend that you use the default value: nolock,tcp,noresvport.
   Job Dependency	Specifies whether the job depends on other jobs. If you want to add job dependencies, enter the job ID and select a dependency.

What to do next

After you submit a job, you can view the job details and the elastic container instances that the system creates.

• In the E-HPC console, go to the Operation page, select the target cluster, and then click the Jobs tab. On this page, you can search for jobs by status, time range, and other criteria.

• On the Container Group page of the Elastic Container Instance console, you can view the elastic container instances that are created for the jobs.