Submitting a job

You can submit jobs to BatchCompute using the command line interface (CLI), console, or software development kit (SDK). This topic describes how to submit a job to BatchCompute.

Submitting a job

The simplest job

If you have configured the default image, instance type, and network type as described in Preparations, you can submit a job using a simple command.

bcs sub "echo 123"  # Submit a single-task job that runs: echo 123

If you have not configured the default settings for the CLI, you must specify additional options when you submit a job.

bcs sub "echo 123" -t ecs.sn1ne.large -i img-ubuntu-vpc --vpc_cidr_block 192.168.0.0/16

The remaining examples in this topic assume that you have configured the default settings.

Submitting a Python script job

bcs sub "python test.py" -p ./test.py # The -p option packages test.py into worker.tar.gz and uploads it to OSS before the job is submitted.

Submitting a job for a directory

This is useful when the src directory contains multiple files, such as:

src
  |-- test.py
  |-- dep.py

If test.py depends on dep.py, you can package the entire directory.

bcs sub "python test.py" -p ./src/  # Package all files in the src directory into worker.tar.gz, upload it to OSS, and then submit the job.

You can also specify multiple files separated by commas:

cd src  # Change to the src directory.
bcs sub "python test.py" -p test.py,dep.py # Package these two files into worker.tar.gz.

If you are not in the src directory, submit the job as follows:

bcs sub "python test.py" -p src/test.py,src/dep.py # Package these two files into worker.tar.gz.

Then, use the following command to view the contents of worker.tar.gz:

tar -tvf worker.tar.gz

The format is as follows:

test.py
dep.py

Submitting a job by mounting the task program

If you upload test.py to oss://mybucket/test/test.py, you can mount oss://mybucket/test/ to a local directory on the virtual machine (VM), such as /home/admin/test/. Then, you can run the program using the "python /home/admin/test/test.py" command.

Submit the job as follows:

bcs sub "python /home/admin/test/test.py" -r oss://mybucket/test/:/home/admin/test/

The parameter -r oss://mybucket/test/:/home/admin/test/ specifies a read-only mount. It mounts oss://mybucket/test/ to /home/admin/test/.

This way, you do not need to package the files into worker.tar.gz.

Other tips

Mounting input data

Assume that your data is uploaded to the oss://my-bucket/inputs/ directory.

bcs sub "python test.py" -p ./src/  -r oss://my-bucket/inputs/:/home/admin/inputs/

• -r specifies a read-only mount. This option mounts the OSS directory oss://my-bucket/inputs/ to /home/admin/inputs/. Your program can then read files in the /home/admin/inputs/ directory as if they were local files.

• Mount the directory to a subdirectory of /home/. Do not mount to system directories such as /bin or /usr.

• To mount multiple directories, separate them with commas. For example: -r oss://my-bucket/inputs/:/home/admin/inputs/,oss://my-bucket/inputs2/:/home/admin/inputs2/

For a Windows image, use the following command to mount the directory:

bcs sub "python test.py" -p ./src/  -r oss://my-bucket/inputs/:D

Automatically uploading results using a mount

My program writes its output to the /home/admin/outputs/ directory, and I want to upload all of its contents to OSS.

bcs sub "python test.py" -p ./src/ -r oss://my-bucket/inputs/:/home/admin/inputs/ -w oss://my-bucket/outputs/:/home/admin/outputs/

• The -w flag specifies a writable mount. After the program is complete, the system automatically uploads the data in this directory to the corresponding OSS directory.

• Do not mount to system directories, such as /bin or /usr. Instead, mount to a directory under /home/.

• To mount multiple directories, separate them with commas. For example: -w oss://my-bucket/outputs/:/home/admin/outputs/,oss://my-bucket/outputs2/:/home/admin/outputs2/

Using custom images and instance types

bcs sub "python test.py" -p ./src/ -c img=img-ubuntu-vpc:type=ecs.sn1ne.large

• -c specifies the cluster. You can specify a cluster ID or an AutoCluster configuration. The format for an AutoCluster configuration is img=${ImageId}:type=${InstanceType}.

• img=img-ubuntu-vpc specifies the image. You can also specify a custom image.

• type=ecs.sn1ne.large specifies the instance type. You can run bcs it to view a list of supported instance types.

• You can specify only -c img=${ImageId} or only -c type=${InstanceType}. If you do not specify a value, the default image and default instance type are used.

Using a cluster

bcs sub "python test.py" -p ./src/ -c cls-xxxxxxx

• -c cls-xxxxxxx specifies the cluster ID cls-xxxxxxx.

For more information, see Using clusters.

Using Docker

bcs sub "python test.py" -p ./src/ --docker myubuntu@oss://my-bucket/dockers/

• myubuntu is shorthand for localhost:5000/myubuntu, and oss://my-bucket/dockers/ is the path to the OSS Docker image repository.

• For more information, see Use Docker.

Custom Disk

bcs sub "python test.py" --disk system:cloud_efficiency:50,data:cloud_efficiency:200

• This parameter applies only to AutoCluster. You can configure a system disk and an optional data disk, separated by a comma. For example: —disk system:cloud_efficiency:40,data:cloud_efficiency:50:/home/disk1. You can also specify only the system disk or only the data disk. For example: —disk system:cloud_efficiency:40

• By default, a 40 GB system disk is mounted.

• System disk configuration format: system:<cloud_efficiency|cloud_ssd>:<40-500>. For example, `system:cloud_efficiency:40` mounts a 40 GB ultra disk as the system disk.

• Data disk configuration format: data:<cloud_efficiency|cloud_ssd>:<5-2000>:<mount_point>. For example, `data:cloud_ssd:50:/home/disk1` mounts a 50 GB standard SSD to /home/disk1 as a data disk. On a Windows instance, the disk can be mounted only to a drive letter. For example, to mount the disk to drive E, use `data:cloud_ssd:50:E`.

Note: For legacy dedicated instances that start with 'bcs', set the disk type to 'ephemeral'. The size of the data disk must be between 5 GB and 1024 GB.

View jobs

For more information, see How to view jobs.