Quickly implement AIGC image generation with the Deepytorch accelerator

Create an ECS instance

The ai-inference-solution Marketplace image used in this topic includes the following three pre-installed models and their runtime environments.

• v1-5-pruned-emaonly.safetensors: The Stable Diffusion v1.5 model, a latent text-to-image diffusion model that can generate photorealistic images from any text input.

  Note

  This model does not perform well with Chinese prompts. We recommend using English prompts.

• Taiyi-Stable-Diffusion-1B-Chinese-v0.1: The Taiyi Chinese model, trained on a filtered dataset of 20 million Chinese image-text pairs, enabling AI image generation using Chinese prompts.

• Taiyi-Stable-Diffusion-1B-Anime-Chinese-v0.1: The Taiyi Anime-style model, the first open-source Chinese Stable Diffusion model for anime. This model was further trained based on Taiyi-Stable-Diffusion-1B-Chinese-v0.1, using a dataset of one million filtered anime-style Chinese image-text pairs. It not only generates high-quality anime images but also retains the strong understanding of Chinese concepts from the original Taiyi Chinese model.

1. Go to the instance creation page.

2. Follow the on-screen instructions to configure the parameters and create an ECS instance.

   Note the following parameters. For information about other parameters, see Create an instance by using the wizard.

   • Instance: Select ecs.gn7i-c16g1.4xlarge.

   • Images: This topic uses a Marketplace image with the inference environment pre-configured. Select the image named ai-inference-solution.

   • Public IP Address: Select Assign Public IPv4 Addresses. Set the bandwidth billing mode to Pay-by-traffic and the peak bandwidth to 10 Mbps.

     Note

     If you need to download models for testing, we recommend setting the peak bandwidth to 100 Mbps to speed up the download.

3. Add a security group rule for the ECS instance. For detailed instructions, see Add a security group rule.

   Set the direction to Inbound, the Port Range to 5000/5000, and the Authorization Object to the public IP address of the local client that accesses the WebUI service (not the instance's public IP address). For example, if your client's public IP address is 101.200.XX.XX, set the source to 101.200.XX.XX/32.

4. After the instance is created, go to the ECS console to obtain its public IP address.

   Note

   The public IP address is used to access the WebUI service for image generation tests.

Add user authentication for Nginx

The image used in this topic has Nginx pre-installed for login authentication to prevent unauthorized access.

1. Run the following command to create a username and password.

   Note

   Replace ${UserName} with your custom username, such as admin. Replace '${Password}' with your custom password, such as ECS@test1234.

   htpasswd -bc /etc/nginx/password ${UserName} '${Password}'

2. Run the following command to restart Nginx.

   systemctl restart nginx

3. Run the following command to check the status of Nginx.

   systemctl status nginx

   If the output shows that Nginx is active (running), the service is operational.

   (base)@iZbp11cbhpoke:~/chatglm/ChatGLM-6B# systemctl status nginx
   ● nginx.service - A high performance web server and a reverse proxy server
        Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
        Active: active (running) since Mon 2023-04-24 14:51:04 CST; 4ms ago
          Docs: man:nginx(8)
       Process: 2467 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
       Process: 2475 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
      Main PID: 2476 (nginx)
         Tasks: 17 (limit: 72274)
        Memory: 14.5M
        CGroup: /system.slice/nginx.service
                ├─2476 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
                ├─2478 nginx: worker process
                ├─2479 nginx: worker process
                ├─2480 nginx: worker process
                ├─2481 nginx: worker process
                ├─2482 nginx: worker process
                ├─2483 nginx: worker process
                ├─2484 nginx: worker process
                ├─2485 nginx: worker process
                ├─2486 nginx: worker process
                ├─2487 nginx: worker process
                ├─2488 nginx: worker process
                ├─2489 nginx: worker process
                ├─2490 nginx: worker process
                ├─2492 nginx: worker process
                ├─2493 nginx: worker process
                └─2494 nginx: worker process

4. Run the following command to enable Nginx to start automatically on boot.

   systemctl enable nginx

Generate images from text

Step 1: Start the WebUI service

Run the following commands to start the WebUI service.

cd ~/stable-diffusion-webui/
nohup ./run_taiyi.sh &

Step 2: Test image generation and acceleration

1. In the address bar of your browser, enter http://<ECS public IP address>:5000. In the login dialog box that appears, enter the username and password that you created in Step 1 of the previous chapter and click Log In.

2. Start AI image generation.

   Note

   The first time you generate an image with Deepytorch, or the first time after switching models, takes an additional 30 seconds to load the Deepytorch model.

   • Generate an image with Deepytorch acceleration enabled (enabled by default)

     Enter a prompt in the text box, such as Armored horses and frozen rivers enter my dreams, concept art, sci-fi, fantasy, 3D, and click Generate. You can click Generate multiple times for different results. The generated image and its inference time appear on the right. In this example, the inference time for a single image is 0.91 seconds.

   • Generate an image with Deepytorch acceleration disabled

     1. Click the Settings tab. In the left-side navigation, select AiaccTorch. Uncheck Apply Aiacctorch in Unet to speedup the whole network inference when loading models, click Apply settings, and then click Reload UI.

     2. Enter a prompt in the text box and click Generate to generate the image again. Check the inference time, which is 2.02 seconds in this example.

   The comparison shows that the inference time for a single image is significantly shorter when Deepytorch Inference is enabled.

Test the inference results of different models

The image used in this topic includes three pre-installed models. You can switch between them to compare their inference results.

1. In the upper-left corner of the page, switch the model. For example, switch to the Taiyi-Stable-Diffusion-1B-Anime-Chinese-v0.1 model.

2. Enter a prompt and a negative prompt in the text boxes.

   • Example prompt:

     1girl, green eyes, baseball cap, blonde hair, closed mouth, hat, looking at viewer, short hair, simple background, solo, upper body, t-shirt

   • Example negative prompt:

     watercolor, comic, scan, simple painting, anime screenshot, 3D, pixel art, original art, sketch, hand-drawn, pencil

     The generated anime-style image is shown below.

Use the LoRA plugin

LoRA is a plugin for the Stable Diffusion model that lets you train a specific style, IP, or character with a small amount of data without modifying the base model. Compared to training a full Stable Diffusion model, LoRA requires significantly fewer resources, making it ideal for community users and individual developers.

You can add the LoRA support text <lora:iuV35.uv1P:1> to a prompt to enable the feature.

1girl, green eyes, baseball cap, blonde hair, closed mouth, hat, looking at viewer, short hair, simple background, solo, upper body, t-shirt, <lora:iuV35.uv1P:1>

Use the ControlNet plugin

ControlNet is a plugin that helps control AI image generation. It uses edge features, depth features, or human pose skeletons from an input image, along with a text prompt, to precisely control the generation process for better visual results.

Canny is a popular model within ControlNet that detects edge information in an input image. It generates a line art sketch from an uploaded image and then uses a prompt to generate a new image with a similar composition.

1. Click the icon next to ControlNet, check Enable, select Canny for the Control type, and upload an input image (such as the Lena image) in the Single Image area.

2. Click Generate. The line art generated by the Canny model and the new image with the same composition appear on the right.