DocsICP FilingConsole
Documentation

Introduction

Updated at:
Copy as MD
Product
My Favorites

BatchCompute lets you deploy applications using Docker images. You can also install software directly on Elastic Compute Service (ECS) images.

You can create a custom Docker image and upload it to an Alibaba Cloud Container Registry repository or to Alibaba Cloud Object Storage Service (OSS) using the registry tool. You can then specify that your job's tasks run in this image.

1. How BatchCompute supports Docker

This section compares the VM method and the Docker method in AutoCluster mode.

  • VM method: You submit a job that can have multiple tasks. Each task specifies an image that supports Linux or Windows. When a task runs, the system starts a virtual machine (VM) from the specified image. The task runs on this VM. After the task is complete, the results are uploaded to persistent storage. The VM is then destroyed, and the system prepares for the next task.

  • Docker method: You submit a job. The system first starts a VM that uses a Docker-enabled OS image, such as Ubuntu. Then, the system downloads your specified Docker image from a Container Registry repository or OSS. The image starts in the VM, and your task runs inside this Docker container. After the task is complete, the results are uploaded to persistent storage. The VM is then destroyed, and the system prepares for the next task.

Note

Currently, one VM can run only one Docker image.

# Using a VM:
---
  |-- job
       |-- task
             |-- VM (User-specified VM, supports Windows and Linux)
                  |-- program (User program)

# Using Docker mode:
---
  |-- job
       |-- task
             |-- VM (Docker-enabled Ubuntu)
                 |-- Docker-Container (User-specified Docker container image)
                      |-- program (User program)

2. Differences between using Docker and not using Docker

-

Without Docker

Using Docker

Image usage

Specify an ECS image ID.

Specify the ID of a Docker-enabled ECS image, such as the official Ubuntu image. You must also specify a custom Docker image.

Application runtime platform

Supports Windows and Linux.

Supports Linux.

Local testing

Local testing is not supported.

Build images locally. Local testing is supported.

3. Install Docker

A) Download and install Docker from the official website.

  • On Windows or macOS, install Toolbox.

After installation, two shortcuts are created:

Note

Kitematic: A graphical user interface (GUI) for managing Docker containers. Docker Quickstart Terminal: A terminal for quickly launching the Docker command-line interface (CLI).

Note

Make sure that your Docker version is 1.10 or later. Earlier versions may have compatibility issues.

B) Configure an accelerator

You can use an accelerator to download official Docker images faster within China. For instructions, see the Alibaba Cloud Container Registry developer platform.

4. Docker usage notes

  • When a Docker container runs, the user is root. The default PATH environment variable is /sbin:/usr/sbin:/bin:/usr/bin. Note that /usr/local/bin is not included in the path.

  • If the PWD environment variable is not set, the default working directory is /batchcompute/workdir. Your program packages are always extracted to /batchcompute/workdir.

  • When you submit tasks with a ClusterID, all jobs in the cluster must have the same BATCH_COMPUTE_DOCKER_REGISTRY_OSS_PATH. This is because the Docker registry runs continuously after it starts.

  • Currently, InputMapping and OutputMapping cannot be mounted to the same directory.

  • BatchCompute starts Docker containers with the --privileged=false flag. Therefore, you cannot start a Docker container from within another Docker container.