DocsICP FilingConsole
官方文档
首页

Manually deploy ChatGLM-6B with the EAIS package

更新时间：
复制 MD 格式
产品详情
我的收藏

By attaching an EAIS instance to an ECS instance (non-GPU), you can create a GPU-accelerated computing environment. Compared to a dedicated GPU cloud server, this method provides elastic GPU resources and reduces both deployment and operational costs. For example, you can attach an EAIS instance on-demand to provide GPU acceleration for your ECS instance. This topic describes how to attach an EAIS instance to an ECS instance, install the EAIS package, and then manually set up a ChatGLM-6B web service and perform model fine-tuning.

Background information

ChatGLM-6B is a conversational language model. You can deploy it by using an EAIS instance or on a dedicated GPU cloud server. Deploying ChatGLM-6B on an ECS instance with an attached EAIS instance is more cost-effective than using a dedicated GPU cloud server. For more information about the benefits of EAIS, see Low cost.

Note

  • Reduced deployment costs:

    You create the EAIS instance only after you finish setting up the environment on your ECS instance. During the setup phase, you are only billed for the non-GPU ECS instance. This is less expensive than setting up the environment on a dedicated GPU cloud server from the start.

  • Reduced usage costs:

    • Flexible GPU attachment: After you deploy ChatGLM-6B, you can go to the EAIS console to detach and release the EAIS instance if you do not need to run ChatGLM-6B tasks. When you need to run tasks again, you can create and attach a new EAIS instance without affecting the environment on your ECS instance. This approach allows you to maintain a less expensive ECS instance and attach GPU resources on demand, which significantly saves costs.

      Note

      For EAIS instances created in the EAIS console or by calling the CreateEaisEi API, you can also stop and start them to flexibly manage GPU capabilities for your ECS instance. You are billed only when the EAIS instance is running. Billing stops when the instance is stopped, which further reduces costs.

    • Lower unit price: Compared to a GPU cloud server with similar computing power, an EAIS instance is more cost-effective.

Billing

The following resources are billed when you deploy and use ChatGLM-6B:

  • ECS instances are billed. For more information, see Billing overview.

  • EAIS instances are billed. For more information, see Pricing.

Environment overview

This topic demonstrates how to deploy ChatGLM-6B with an EAIS instance, using the following ECS and EAIS instance configurations as an example.

  • ECS instance:

    • Instance type: ecs.g7.xlarge (4 vCPU, 16 GiB memory)

    • Operating system: Ubuntu 20.04

    • System disk capacity: 100 GiB

  • EAIS instance type: eais.ei-a6.2xlarge

  • Region and availability zone for the ECS and EAIS instances: China (Beijing) Zone I

    Note

    Placing the ECS instance and the EAIS instance in the same availability zone improves inference performance. You can create an ECS instance in any region and availability zone that supports your desired EAIS instance type. For a list of supported regions and availability zones for EAIS instance types, see Instance type limitations.

Step 1: Create an ECS instance

  1. Log in to the Alibaba Cloud console and go to the ECS Instance Creation page.

  2. Create a Virtual Private Cloud (VPC) type ECS instance to which you will attach the EAIS instance.

    For detailed instructions, see Create an instance.

    When you create the ECS instance, take note of the following configurations. You can configure other settings based on your business requirements.

    Parameter

    Example

    Billing method

    pay-as-you-go

    Region

    China (Beijing)

    Network and zone

    • Network Type: VPC

    • Availability Zone: China (Beijing) Zone I

    Instance type

    ecs.g7.xlarge

    Image

    Select Public Image, and then select Ubuntu 20.04 64-bit.

    System disk

    Set Capacity to 100 GiB.

    Public IP address

    Select Assign Public IPv4 Address.

Step 2: Set up the ChatGLM-6B environment

  1. Remotely connect to the ECS instance.

    For detailed instructions, see Connect to an instance by using a password on a Linux or macOS client.

  2. Run the following commands to install CUDA 11.8.0.

    wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
sh cuda_11.8.0_520.61.05_linux.run --silent --toolkit

  3. Install the eais-tool package and view information about the EAIS instance.

    For detailed instructions, see eais-tool.

  4. Install the eais-cuda package.

    For detailed instructions, see eais-cuda.

  5. Run the following commands to download the ChatGLM-6B model.

    The ChatGLM-6B model is approximately 25 GB, and the download may take 15 to 30 minutes.

    Note

    Because the ChatGLM-6B model is large, automatic download by the program can be slow. We recommend that you run these commands to download the model in advance.

    apt-get update
apt-get install -y git git-lfs
git clone https://huggingface.co/THUDM/chatglm-6b

  6. Run the following commands to download the ChatGLM-6B source code and install the ChatGLM-6B runtime environment.

    git clone https://github.com/THUDM/chatglm-6b
cd ChatGLM-6B
pip3 install -r requirements.txt

Step 3: Create and attach an EAIS instance

  1. Create an EAIS instance.

    For detailed instructions, see Create an EAIS instance.

    When you create the EAIS instance, take note of the following configurations. You can configure other settings based on your business requirements.

    Parameter

    Example

    Region

    China (Beijing)

    Instance type

    eais.ei-a6.2xlarge

  2. Attach the EAIS instance to the ECS instance that you created in Step 1: Create an ECS instance.

    For detailed instructions, see Attach an EAIS instance to an ECS instance.

Step 4: Start the EAIS instance

Important

This step is required only for EAIS instances created by using the EAIS console or the CreateEaisEi API.

After binding the EAIS instance to the ECS instance, you must start the EAIS instance to use its GPU capabilities. For detailed instructions, see Start an EAIS instance.

Step 5: Use ChatGLM

You can use EAIS with ChatGLM to set up a web service or to fine-tune your model. Follow the procedure for your desired task.

Web service

If you need to set up a web service with ChatGLM, follow these steps.

  1. Remotely connect to the ECS instance.

    For detailed instructions, see Connect to an instance by using a password on a Linux or macOS client.

  2. Find and modify the web_demo.py script in the ChatGLM-6B source code that you downloaded in Step 2: Set up the ChatGLM-6B environment.

    1. Replace THUDM/chatglm-6b in the script with the local path to the chatglm-6b model directory. This topic uses /root/chatglm-6b as an example.

      The following shows an example of the script to modify:

      from transformers import AutoModel, AutoTokenizer
import gradio as gr
import mdtex2html
tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).half().cuda()
model = model.eval()
"""Override Chatbot.postprocess"""

    2. Change share=False to share=True in the script.

      The following shows an example of the code to modify:

      history = gr.State([])
submitBtn.click(predict, [user_input, chatbot, max_length, top_p, temperature, history], [chatbot, history],
                show_progress=True)
submitBtn.click(reset_user_input, [], [user_input])
emptyBtn.click(reset_state, outputs=[chatbot, history], show_progress=True)
demo.queue().launch(share=False, inbrowser=True)

  3. Run the following command to start the ChatGLM-6B web service.

    python3 web_demo.py

    If the following output is returned, the ChatGLM-6B web service has started successfully. Take note of the public URL.

    root@iZ...Z:~/ChatGLM-6B# python3 web_demo.py
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 100%|██████████████████████████████████████████████████| 8/8 [01:40<00:00, 12.60s/it]
Running on local URL:  http://127.0.0.1:7860
Running on public URL: https://1d9b1eb4025cc2fc2c.gradio.live
This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces

  4. On your local machine, open a browser and navigate to the public URL to check if the ChatGLM-6B web service was successfully set up.

    For example, if the public URL is https://1d9b1eb4025cc2fc2c.gradio.live, the following page appears. This confirms that the ChatGLM-6B web service is set up successfully, and the chat interface is displayed. The left side contains the dialogue input area and the Submit button. On the right side, you can adjust the generation parameters: Maximum length (default: 2048), Top P (default: 0.7), and Temperature (default: 0.95).

    You can now test the service. For details, see Web Demo.

Fine-tuning

You can use ChatGLM to fine-tune your model. For detailed instructions, see ChatGLM-6B. Note the following modifications required for this procedure:

Note

For more information about fine-tuning, see ptuning.

Before fine-tuning, find the train.sh script in the ChatGLM-6B source code that you downloaded in Step 2: Deploy the ChatGLM-6B environment on an ECS instance. This script is located in the ptuning directory. Then, in the train.sh script, replace THUDM/chatglm-6b with the path to your local chatglm-6b model folder, for example, /root/chatglm-6b. In the script, the value of the --model_name_or_path parameter, THUDM/chatglm-6b, must be replaced with the actual model path.

PRE_SEQ_LEN=128
LR=2e-2
CUDA_VISIBLE_DEVICES=0 python3 main.py \
    --do_train \
    --train_file AdvertiseGen/train.json \
    --validation_file AdvertiseGen/dev.json \
    --prompt_column content \
    --response_column summary \
    --overwrite_cache \
    --model_name_or_path THUDM/chatglm-6b \
    --output_dir output/adgen-chatglm-6b-pt-$PRE_SEQ_LEN-$LR \
    --overwrite_output_dir \
    --max_source_length 64

(Optional) Step 6: Stop the EAIS instance

Important

This operation is supported only for EAIS instances created by using the EAIS console or the CreateEaisEi API.

When you temporarily do not need the GPU capabilities of an EAIS instance, you can stop the instance to avoid unnecessary charges. For detailed instructions, see Stop an EAIS instance.

(Optional) Step 7: Detach and release the EAIS instance

For greater business flexibility, you can detach an EAIS instance from the ECS instance and then release it. This enables elastic use of GPU resources.

  1. Detach the EAIS instance from the ECS instance.

    For detailed instructions, see Detach an EAIS instance from an ECS instance.

  2. Release the EAIS instance.

    For detailed instructions, see Release an EAIS instance.

Related topics

Previous：Quick start: Deploy ChatGLM-6B with an EAIS imageNext：Deploy ChatGLM2-6B by using an EAIS instance