This topic describes how to create an accelerated image to reduce the time between job submission and execution in an auto scaling cluster.
Background
In auto scaling clusters, a significant delay can occur between job submission and execution. This delay is mainly caused by the time required for scaling decisions and actions to complete.
By pre-installing the Python environment and its dependencies in the image, you shorten the startup time for new nodes and the overall job execution time.
Prerequisites
-
This feature is currently supported only in the China (Hangzhou) region.
-
The cluster status is Running.
-
Upgrade the cluster client (ehpcutil) to version
2.0.47or later.NoteClick the name of the target cluster to go to the Cluster Configuration page. On this page, you can check the client version and perform an upgrade. If no upgrade option is available, your client is already up to date.
Procedure
Step 1: Create an instance
-
Go to the instance purchase page and select the Custom Launch tab. Configure the instance to meet the following requirements. For more information, see Custom purchase of an instance.
-
Billing Method: Select Pay-as-you-go.
-
Instance Type: Select an instance type with the x86_64 architecture.
-
Image: Select an image from the CentOS 7 series.
-
Public IP Address: Select Assign Public IPv4 Address.
-
Security Group: Select the security group that your cluster uses.
-
-
Click Create Order to create the instance.
Step 2: Install environments
-
After the instance status changes to Running, connect to the ECS instance that you created. For more information, see Connect to a Linux instance by using Workbench.
-
Download and install the required Python environments.
-
In the root directory, create an installation script named
install_python.sh.#!/bin/bash PYTHON_INSTALL_ROOT=/usr/local/py3.9env SERVICE_INSTALL_ROOT=/usr/local/ehpc_service mkdir $PYTHON_INSTALL_ROOT set -e arch=`arch` srcEnvPath=$1 # Pass this as an argument when calling the script #py3=$2 # Pass this as an argument (0 or 1) to decide which Python version to install pythonenv="python3.9env" echo "Downloading and extracting Python 3.9.10 for ${arch}..." wget -c -nv ${srcEnvPath}/python3.9.10.$arch.tgz -O - | tar -xz -C /usr/local/py3.9env/ echo "Downloading and extracting ${pythonenv} for ${arch}..." curl -s -O ${srcEnvPath}/${pythonenv}.$arch.tgz if [ -d /usr/local/py3.9env/${pythonenv} ]; then rm -rf /usr/local/py3.9env/${pythonenv} fi tar -xzf ${pythonenv}.$arch.tgz -C /usr/local/py3.9env/ echo "Downloading and extracting Python 2.7 environment..." curl -s -O ${srcEnvPath}/centos/7.2/python2.7env.tgz if [ -d /usr/local/python2.7env ]; then rm -rf /usr/local/python2.7env fi tar -xzf python2.7env.tgz -C /usr/local/ -
Grant execute permissions to the script.
chmod +x install_python.sh -
Run the script to download and install the environments.
bash /root/install_python.sh http://public-ehs.vpc100-oss-cn-hangzhou.aliyuncs.com/packages/env -
Verify that the py3.9env and python2.7env directories were created in the
/usr/local/directory to confirm a successful installation.
-
-
Download and install Node.js and other dependency packages.
-
In the root directory, create a download and installation script named
install_lib.sh.#!/bin/bash set -e arch=`arch` # Pass this as an argument when calling the script srcEnvPath=$1 # Pass this as an argument when calling the script # echo "Installing Node.js for aarch64..." # curl -s -O $srcEnvPath/node-v16.17.1-linux-arm64.tar.xz # tar -xf node-v16.17.1-linux-arm64.tar.xz # mv node-v16.17.1-linux-arm64 /usr/local/node-v16.17.1-linux-arm64 # ln -sfT /usr/local/node-v16.17.1-linux-arm64/bin/node /usr/local/bin/node # ln -sfT /usr/local/node-v16.17.1-linux-arm64/bin/npm /usr/local/bin/npm echo "Installing Node.js for x86_64..." curl -s -O $srcEnvPath/node-v6.11.1-linux-x64.tar.gz tar -xzf node-v6.11.1-linux-x64.tar.gz mv node-v6.11.1-linux-x64 /usr/local/node-v6.11.1 ln -sfT /usr/local/node-v6.11.1/bin/node /usr/local/bin/node ln -sfT /usr/local/node-v6.11.1/bin/npm /usr/local/bin/npm echo "Installing additional libraries..." yum clean all && yum makecache yum install -y perf postgresql-contrib hwloc-devel jsoncpp-devel libcurl-devel nfs-utils cifs-utils environment-modules unzip jq echo "Appending module path..." echo -e "\nexport MODULEPATH=/opt/ehpcmodulefiles" >> /etc/bashrc kernel_version="$(uname -r)" kernel_devel_pkg="kernel-devel-${kernel_version}" base_os="7" kernel_devel_remote_repo="http://mirrors.cloud.aliyuncs.com/opsx/ecs/linux/rpm/driver/${base_os}/x86_64" rpm -q "${kernel_devel_pkg}" || yum install -y "${kernel_devel_pkg}" echo "Installing xdragon-health-protector..." xdragon_pkg="xdragon-health-protector-vm-1.0.x86_64.rpm" curl -s -O $srcEnvPath/${xdragon_pkg} yum install -y ${xdragon_pkg} systemctl enable xdragon-health-protector systemctl restart xdragon-health-protector systemctl status xdragon-health-protector -
Grant execute permissions to the script.
chmod +x install_lib.sh -
Run the installation script. A
runningstatus indicates a successful installation.bash /root/install_lib.sh http://public-ehs.vpc100-oss-cn-hangzhou.aliyuncs.com/packages/env
-
Step 3: Create a custom image
-
Go to the ECS console - Instances page and click the ID of the instance you created to go to the Instance Details page.
-
Click Create a custom image to generate the accelerated image.

Step 4: Configure cluster auto scaling
-
Log on to the E-HPC console. Click the target Cluster Name to go to the Cluster Details page.
-
In the left-side navigation pane, choose . In the Operation column of the target queue, click Edit.
-
On the Edit Queue page, configure the parameters for the Instance type Group. For more information, see Configure an auto-scaled node.
-
Click Add an instance specification and select the instance types for the nodes.
-
Click Custom image and select the accelerated image that you created.

-
Click OK.
-
-
Click Save to apply the auto scaling configuration.