Getting Started with DAG Jobs

If you have not activated the BatchCompute service, activate it first.

Install and configure the command line interface

For more information, see Install and configure the command line interface.

Job preparation

This job calculates a sum by adding all the numbers in input.txt and writing the result to output.txt.

This job requires only one task because the calculation is simple.

This example mounts an OSS directory as a local directory on a virtual machine (VM) to perform file-based operations.

Upload the data file to OSS

First, create an `input.txt` file.

Content of `input.txt` (ensure one number per line):

2
40
51

Upload `input.txt` to your OSS bucket:

bcs o up input.txt oss://your-bucket/sum/inputs/

# Check after the upload is complete
bcs o ls oss://your-bucket/sum/inputs/

• Replace `your-bucket` with the name of your bucket. This example assumes the region is `cn-shenzhen`.

• The bcs o command provides several common functions for OSS. Run bcs o -h to view the help documentation. This command is convenient for testing small amounts of data. However, do not use it to upload or download large amounts of data. Because the command is not multi-threaded, uploads and downloads are slow. For more information about how to upload data to OSS, see Common OSS Tools.

Prepare the task program

Content of `sum.sh`:

#!/bin/bash
t=0
while read LINE
do
 t=$(($t+${LINE}))
done < /home/inputs/input.txt
echo $t
echo $t > /home/outputs/output.txt

In the previous step, you uploaded the `input.txt` file to oss://your-bucket/sum/inputs/. The program reads `input.txt` from the /home/inputs/ directory on the VM. This is possible because of the BatchCompute feature that mounts OSS directories. The configuration for this mount is explained in the next step, "Submit the job".

This example uses a bash script to perform the summation. You can also run other applications from the script. To learn how to deploy applications to the BatchCompute environment, see Custom images and Use Docker.

Submit the job

Run the following command from the directory that contains `sum.sh` to submit the job:

bcs sub "sh sum.sh" -p sum.sh -r oss://your-bucket/sum/inputs/:/home/inputs/ -w oss://your-bucket/sum/outputs/:/home/outputs/

• This command uses the default image and default instance type.

• The `-r` option specifies a read-only mount. It mounts the OSS directory oss://your-bucket/sum/inputs/ to the local VM directory /home/inputs/. The program can then access the `input.txt` file from the /home/inputs/ path.

• The `-w` option specifies a writable mount. It mounts the OSS directory oss://your-bucket/sum/outputs/ to the local VM directory /home/outputs/. After the program runs, the `output.txt` file written to the local `/home/outputs/` directory is automatically uploaded to the corresponding OSS directory.

View job status and results

bcs j   # Get and cache the job list. The first job is usually the one you just submitted.
bcs j 1  # View the details of the first job in the cache
bcs log 1 # View the log of the first job in the cache

Run the following command to view the results:

bcs o cat oss://your-bucket/sum/outputs/output.txt