Deploy ChatGLM2-6B by using an EAIS instance

Background

ChatGLM2-6B is an open-source conversational language model. You can deploy ChatGLM2-6B by using an EAIS instance or directly on a GPU instance. Attaching an EAIS instance to an ECS instance to deploy ChatGLM2-6B is more cost-effective than directly purchasing a GPU instance. For more information about the benefits of EAIS instances, see Cost-effectiveness.

• Reduced deployment costs:

  You can complete the environment setup on your non-GPU ECS instance before creating an EAIS instance. During this phase, you are billed only for the ECS instance. This approach is more cost-effective than setting up the environment directly on a GPU instance.

• Reduced usage costs:

  • Flexible GPU resource attachment: After deploying ChatGLM2-6B, you can detach and release the EAIS instance from the console when you do not need to run tasks. Later, you can create and attach a new EAIS instance without affecting the environment on your ECS instance. This allows you to run an inexpensive ECS instance long-term and attach GPU resources only when needed, which saves costs.

    Note

    For EAIS instances created in the console or by using the CreateEaisEi operation, you can flexibly manage GPU resources by starting and stopping the instances. You are charged for an EAIS instance only while it is running, not when it is stopped, further reducing costs.

  • An EAIS instance is less expensive than a GPU instance with the same computing power.

Billing

Deploying and using ChatGLM2-6B incurs charges for the following resources:

• ECS instances are billed. For more information, see Billing overview.

• EAIS instances are billed. For more information, see Billing.

Environment

This topic demonstrates the deployment of ChatGLM2-6B using the following EAIS and ECS instance configurations.

• ECS instance:

  • Instance type: ecs.g7.xlarge (4 vCPU, 16 GiB memory)

  • Operating system: Ubuntu 20.04

  • System disk capacity: 100 GiB

• EAIS instance type: eais.ei-a6.2xlarge

• Region and availability zone for the ECS and EAIS instances: China (Beijing) Zone I

  Note

  Placing the ECS instance and the EAIS instance in the same availability zone improves inference performance. You can create an ECS instance in any region and availability zone that supports your desired EAIS instance type. For a list of supported regions and availability zones for EAIS instance types, see Instance type limitations.

Step 1: Create an ECS instance

1. Log on to the ECS instance buy page.

2. Create a VPC-type ECS instance to which you want to attach an EAIS instance.

   For more information, see Create an instance.

   When you create the ECS instance, take note of the following parameters. You can use the default values for other parameters.

   Parameter	Example configuration
   Billing method	pay-as-you-go
   Region	China (Beijing)
   Network and zone	Network Type: VPC Availability Zone: Zone I
   Instance type	ecs.g7.xlarge
   Image	Select Public image and set the image to Ubuntu 20.04 64-bit.
   System disk	Set Capacity to 100 GiB.
   Public IP	Select Assign Public IPv4 Address.

Step 2: Deploy the ChatGLM2-6B environment

1. Connect to the ECS instance.

   For more information, see Connect to a Linux or macOS instance by using a password.

2. Install Docker.

   1. Run the following command to install Docker dependencies:

      apt-get update && \
      apt install -y --no-install-recommends software-properties-common

   2. Run the following command to install Docker:

      curl -fsSL https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/gpg | apt-key add -
      add-apt-repository -y \
      "deb [arch=amd64] https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu \
      $(lsb_release -cs) \
      stable" && \
      apt-get update && \
      apt-get install -y docker-ce

3. Run the following command to pull the ChatGLM2-6B image provided by EAIS:

   docker pull registry.cn-beijing.aliyuncs.com/eai_beijing/chatglm2-6b:v4.2.3

Step 3: Create and attach an EAIS instance

1. Create an EAIS instance.

   For more information, see Create an EAIS instance.

   When you create the EAIS instance, configure the following parameters. Configure other parameters based on your business needs.

   Parameter	Example value
   Region	China (Beijing)
   Instance Type	eais.ei-a6.2xlarge

2. Attach the EAIS instance to the ECS instance that you created in Step 1: Create an ECS instance.

   For more information, see Attach an EAIS instance to an ECS instance.

Step 4: Start the EAIS instance

After binding the EAIS instance to the ECS instance, you must start the EAIS instance to use its GPU capabilities. For detailed instructions, see Start an EAIS instance.

Step 5: Start the ChatGLM2-6B web service

1. Connect to the ECS instance.

   For more information, see Connect to a Linux or macOS instance by using a password.

2. Run the following command to start the ChatGLM2-6B web service:

   docker run --rm -it --net host registry.cn-beijing.aliyuncs.com/eai_beijing/chatglm2-6b:v4.2.3

   The following output indicates that the ChatGLM2-6B web service has started. Record the port of the ChatGLM2-6B web service. By default, port 7860 is used.

   root@xxx:~# docker run --rm -it --net host registry.cn-beijing.aliyuncs.com/eai_beijing/chatglm2-6b:v4.2.3
   Loading checkpoint shards: 100%|████████████████████████████████████████████████████████| 7/7 [01:05<00:00, 9.36s/it]
   /opt/conda/lib/python3.10/site-packages/gradio/components/textbox.py:259: UserWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
     warnings.warn()
   Running on local URL:  http://0.0.0.0:7860
   To create a public link, set `share=True` in `launch()`.

3. Configure a security group rule for the ECS instance.

   You must add a security group rule to allow access to port 7860. Set the Source parameter to the IP address or CIDR block from which you will access the ChatGLM2-6B web service. For more information, see Add a security group rule.

   On the Inbound tab, add a rule, set Action to allow, Priority to 1, Protocol Type to Custom TCP, and Port Range to 7860/7860.

4. From your local machine, open a browser and navigate to http://the public IP address of the ECS instance:7860 to verify the deployment of the ChatGLM2-6B web service.

   If the Gradio chat UI for ChatGLM2-6B appears, featuring the Chatbot conversation area, an input box, and the Maximum length and Top P sliders, the web service is successfully deployed.

(Optional) Step 6: Stop the EAIS instance

When you temporarily do not need the GPU capabilities of an EAIS instance, you can stop the instance to avoid unnecessary charges. For detailed instructions, see Stop an EAIS instance.

Related documents

• If you want to deploy ChatGLM-6B by using an EAIS instance, see Deploy ChatGLM-6B by using an EAIS instance.

• If you want to deploy Stable Diffusion by using an EAIS instance, see Quickly deploy Stable Diffusion by using an image provided by EAIS.