SGE cluster support

2. Create a custom image (Optional)

This step is optional. If you do not have special requirements for the image, you can skip this step. To install software on top of this OS image, you must create a custom image based on the BatchCompute SGE image..

3. Prepare the SGE Master node

Designate an ECS virtual machine (VM) as the Master node for the SGE system. This node manages the entire cluster and can also act as a job submission node. If you use a custom image, select the custom image when you start the VM. Otherwise, select the BatchCompute SGE image.

For information about parameter configurations, see Create a Linux instance.

Because the Master node must run for a long time, select the subscription billing method when you start the VM. For testing purposes, select the pay-as-you-go billing method.

The steps are as follows:

A) Create a VPC and a vSwitch

• Open the ECS console. Click Virtual Private Cloud to go to the VPC console. Then, click the Virtual Private Cloud menu item.

• Create a VPC. In this example, set the VPC CIDR block to 192.168.0.0/16 and the vSwitch CIDR block to 192.168.0.0/24.

• Create a vSwitch.

B) Purchase an ECS VM

• Select the VPC that you created. Click vSwitch to go to the vSwitch list. Then, click Create Instance to go to the ECS instance creation page.

• For Public IP Address, select Allocate.

• Select a security group. A security group was automatically created when you created the VPC. Only one security group is available.

• The instance type must have at least 2 cores and 4 GB of memory.

• Marketplace Image: BatchCompute SGE.

• Set a password.

• Configuration Reference

• Note that you cannot change the instance name after the instance is created.

3. Start the SGE cluster

BatchCompute provides a command-line interface (CLI) tool named bccluster (bcc) to help you manage SGE clusters. This tool is pre-installed in the BatchCompute SGE image.

A) Log on to the Master node

Use the ssh command to log on to the Master node as the root user.

ssh root@<PublicIP>

Then, enter the password that you set when you purchased the ECS instance.

B) Log on with the bccluster command

The bccluster (bcc) tool is used to manage the SGE cluster, including start, scale-out, and stop operations. If this is your first time logging on to the Master node, update the bccluster tool first. Then, run the following command to configure the region, AccessKey ID, and AccessKey secret. The region must be the same as the region where the Master instance is located.

pip install -U batchcompute-sge #If the command fails, retry it.
bcc login <region> <accessKeyId> <accessKeySecret>

• Run this command only the first time you log on to the Master node.

• The RAM user that corresponds to the AccessKey must be granted full permissions for BatchCompute, read permissions for ECS, and permissions for the AuthorizeSecurityGroup and RevokeSecurityGroup ECS write API operations. To grant the permissions, open the RAM console, click the Users menu, and select the RAM user.

• For a list of regions, see Regions and zones.

• If the following error occurs when you run the bcc login command, it indicates that the ECS instance name was changed. You must delete the ECS instance, create a new one, and then log on again.

C) Start the cluster

Start the worker nodes.

bcc start -n 2 -t ecs.sn2.medium -i img-sge --vpc_cidr_block=192.168.1.0/24

Parameters:

• -n specifies the number of worker nodes to start.

• -t specifies the instance type for the worker nodes. The bcc t command lists the available instance types.

• -i specifies the image for the worker nodes. You can specify an OS image ID, such as img-sge, or a custom image ID.

• --vpc_cidr_block specifies the cluster CIDR block. For more information, see Plan CIDR blocks.

After you run the command, a start instruction is submitted. The worker nodes take some time to start. You must wait for the nodes to start before you can use the cluster.

D) Check the BatchCompute cluster status

bcc status

This command shows the cluster status, the startup status of the worker nodes, and other information.

E) Check the SGE cluster status

qhost

Run the qhost command to check whether the SGE cluster has fully started.

4. Release (delete) the SGE cluster

If you no longer need the worker nodes, use the stop command to stop all of them. If you no longer need the Master node, you can delete it in the console.

bcc stop

5. How to start a NAS-mounted SGE cluster

You can use the bcc tool to easily mount a NAS file system. The following example shows how to do this:

bcc start -n 2 -t ecs.sn1.medium -i img-sge --vpc_cidr_block=192.168.1.0/24 -m nas://a/b/c:/home/nas/

6. Launch a cluster with multiple instance types (multiple groups)

When you run the bcc start command, add the `--group_num 4` option to create four groups.

bcc start -n 2 -t ecs.sn2.medium -i img-sge --vpc_cidr_block=192.168.1.0/24 --group_num 4

• The group names are default, group1, group2, and group3. The number of nodes in group1, group2, and group3 is 0.

• You can use the bcc update command to change the instance type for all groups or for a specific group. For more information, run bcc update --help.

• You can use the bcc resize command to change the number of nodes in a specific group. For more information, run bcc resize --help.