Generate HD long videos with EasyAnimate

Solutions

Solution	Advantages and use cases	Billing
Solution 1: Generate videos using DSW	Provides a cloud-based Integrated Development Environment (IDE) with built-in tutorials and code. This solution is ideal for users who want to explore models in depth or perform custom development.	This tutorial creates a DSW instance that uses public resources and is billed on a pay-as-you-go basis. For more information, see DSW Billing.
Solution 2: Generate videos using Model Gallery	No environment configuration is required. You can deploy or fine-tune models with a single click and then call them via a WebUI or API. This solution is ideal for rapid validation or application integration.	This tutorial creates an EAS service for model deployment and a DLC job for model fine-tuning. Both use public resources and are billed on a pay-as-you-go basis. For more information, see DLC Billing and EAS Billing.

Solution 1: Generate videos using DSW

Step 1: Create a DSW instance

Log on to the PAI console and select a region. In the left-side navigation pane, click Workspaces, then find and click the name of your target workspace.
In the left-side navigation pane, click Model Training > Interactive Modeling (DSW).

Click Create instance and configure the following key parameters. Leave the other parameters at their default values.

Parameter	Description
Instance Name	Example: AIGC_test_01.
Resource Type	Select Public Resources.
Instance Type	Under GPU Specs, select `ecs.gn7i-c8g1.2xlarge` or another A10 or GU100 instance type.
Image	Select Alibaba Cloud Image, then search for and select `easyanimate:1.1.5-pytorch2.2.0-gpu-py310-cu118-ubuntu22.04`.

Click OK to create the instance, and wait for its status to change to Running.

Step 2: Download the tutorial and model

In the Actions column of the target DSW instance, click Open to enter the development environment.
On the Notebook tab, open the Launcher page and click DSW Gallery.
In DSW Gallery, search for AI Video Generation with EasyAnimate (V5). Click Open in DSW to download the required resources for this tutorial to the DSW instance.

The AI video generation example for EasyAnimate is available in multiple versions. This tutorial uses V5 as an example.
Download and install the EasyAnimate code and model.

In the EasyAnimate tutorial file, click to sequentially run the cells titled Function Definition, Download Code, and Download Model.

Step 3: Launch the WebUI and generate a video

Click to run the cell titled Launch UI, which starts the WebUI service.

Click the generated link to open the WebUI.

!python app.py
/mnt/workspace/demos/easyanimate_v5_1/EasyAnimate/easyanimate/ui/ui.py:892: GradioUnusedKwargWarning: You have unused kwarg parameters in Video, please remove them: {'sources': 'upload'}
    control_video = gr.Video()
/usr/local/lib/python3.10/dist-packages/gradio/utils.py:842: UserWarning: Expected 2 arguments for function <function ui.<locals>.upload_generation_method at 0x7fe5b79b79a0>, received 1.
    warnings.warn()
/usr/local/lib/python3.10/dist-packages/gradio/utils.py:846: UserWarning: Expected at least 2 arguments for function ui.<locals>.upload_generation_method at 0x7fe5b79b79a0>, received 1.
    warnings.warn()
Running on local URL:  http://0.0.0.0:7860
IMPORTANT: You are using gradio version 3.41.2, however version 4.44.1 is available, please upgrade.
--------
To create a public link, set `share=True` in `launch()`.

In the WebUI, select the path of the pre-trained model and configure other parameters as needed.

From the Pre-trained model path drop-down list, select /mnt/workspace/demos/easyanimate_v5/EasyAnimate/models/Diffusion_Transformer/EasyAnimateV5-12b-zh-InP/.
Click Generate. After about 5 minutes, you can view or download the generated video on the right.

Solution 2: Generate videos using Model Gallery

Step 1: Deploy the pre-trained model

Log on to the PAI console and select a region. In the left-side navigation pane, click Workspaces, then find and click the name of your target workspace.
In the left-side navigation pane, click Quick Start > Model Gallery. Search for EasyAnimate HD Long Video Generation Model, click Deploy, and then confirm the deployment. When the service status changes to Running, the model is successfully deployed.

Step 2: Generate videos using the WebUI or API

After deploying the model, you can generate videos by calling the service via the WebUI or an API.

To view deployment job details later, in the left-side navigation pane, click Model Gallery > Job Management > Deployment Jobs, and then click the Service name.

WebUI

On the Service details page, click View Web App.
In the WebUI, select the path of the pre-trained model and configure other parameters as needed.

For example, in the Pre-trained model path drop-down list, select /mnt/models/Diffusion_Transformer/EasyAnimateV3-XL-2-InP-512x512/.
Click Generate. After about 5 minutes, you can view or download the generated video on the right.

When the generation is complete, the Generation Info area in the lower-right corner displays Success, indicating the video was generated successfully.

API

On the Service details page, in the Resource Details section, click View Call Information to get the service endpoint and token.

Call the service to generate a video. The following Python code provides an example request:

The input parameters for the service interface are as follows:import os
import requests
import json
import base64
from typing import Dict, Any

class EasyAnimateClient:
    """
    API client for the EasyAnimate EAS service.
    """
    def __init__(self, service_url: str, token: str):
        if not service_url or not token:
            raise ValueError("The `service_url` and `token` parameters cannot be empty.")
        self.base_url = service_url.rstrip('/')
        self.headers = {
            'Content-Type': 'application/json',
            'Authorization': token
        }

    def update_model(self, model_path: str, edition: str = "v3", timeout: int = 300) -> Dict[str, Any]:
        """
        Updates and loads the specified model version and path.

        Args:
            model_path: The path to the model within the service, such as "/mnt/models/Diffusion_Transformer/EasyAnimateV3-XL-2-InP-512x512".
            edition: The model version. Defaults to "v3".
            timeout: The request timeout in seconds. A longer timeout is recommended as model loading can be slow.
        """
        # 1. Update the version.
        requests.post(
            f"{self.base_url}/easyanimate/update_edition",
            headers=self.headers,
            json={"edition": edition},
            timeout=timeout
        ).raise_for_status()

        # 2. Update the model path and wait for it to load.
        print(f"Sending request to load model: {model_path}")
        response = requests.post(
            f"{self.base_url}/easyanimate/update_diffusion_transformer",
            headers=self.headers,
            json={"diffusion_transformer_path": model_path},
            timeout=15000
        )
        response.raise_for_status()
        return response.json()

    def generate_video(self, prompt_textbox: str, **kwargs) -> bytes:
        """
        Generates a video based on a prompt.

        Args:
            prompt_textbox: The positive prompt in English.
            **kwargs: Other optional parameters. See the parameter description table below.

        Returns:
            The binary data of the video in MP4 format.
        """
        payload = {
            "prompt_textbox": prompt_textbox,
            "negative_prompt_textbox": kwargs.get("negative_prompt",
                                                  "The video is not of a high quality, it has a low resolution..."),
            "width_slider": kwargs.get("width_slider", 672),
            "height_slider": kwargs.get("height_slider", 384),
            "length_slider": kwargs.get("length_slider", 144),
            "sample_step_slider": kwargs.get("sample_step_slider", 30),
            "cfg_scale_slider": kwargs.get("cfg_scale_slider", 6.0),
            "seed_textbox": kwargs.get("seed_textbox", 43),
            "sampler_dropdown": kwargs.get("sampler_dropdown", "Euler"),
            "generation_method": "Video Generation",
            "is_image": False,
            "lora_alpha_slider": 0.55,
            "lora_model_path": "none",
            "base_model_path": "none",
            "motion_module_path": "none"
        }

        response = requests.post(
            f"{self.base_url}/easyanimate/infer_forward",
            headers=self.headers,
            json=payload,
            timeout=1500
        )
        response.raise_for_status()

        result = response.json()
        if "base64_encoding" not in result:
            raise ValueError(f"Invalid API response format: {result}")

        return base64.b64decode(result["base64_encoding"])


# --- Example Usage ---
if __name__ == "__main__":
    try:
        # 1. Configure the service information. Replace the placeholders with your actual service endpoint and token. We recommend setting them as environment variables.
        EAS_URL = "<eas-service-url>"
        EAS_TOKEN = "<eas-service-token>"

        # 2. Create the client.
        client = EasyAnimateClient(service_url=EAS_URL, token=EAS_TOKEN)

        # 3. Load the model. By default, no model is loaded after service deployment. You must call `update_model` at least once to specify the model to use before sending a generation request. To switch models later, call this method again.
        client.update_model(model_path="/mnt/models/Diffusion_Transformer/EasyAnimateV3-XL-2-InP-512x512")

        # 4. Generate the video.
        video_bytes = client.generate_video(
            prompt_textbox="A beautiful cat playing in a sunny garden, high quality, detailed",
            width_slider=672,
            height_slider=384,
            length_slider=72,
            sample_step_slider=20
        )

        # 5. Save the video file.
        with open("api_generated_video.mp4", "wb") as f:
            f.write(video_bytes)
        print("Video saved successfully as api_generated_video.mp4")

    except requests.RequestException as e:
        print(f"Network request error: {e}")
    except (ValueError, KeyError) as e:
        print(f"Data or parameter error: {e}")
    except Exception as e:
        print(f"An unknown error occurred: {e}")

API parameter description

Parameter	Description	Type	Default
prompt_textbox	A positive prompt that describes the desired video content.	string	Required. No default value.
negative_prompt_textbox	A negative prompt that specifies what to exclude from the video.	string	"The video is not of a high quality, it has a low resolution, and the audio quality is not clear. Strange motion trajectory, a poor composition and deformed video, low resolution, duplicate and ugly, strange body structure, long and strange neck, bad teeth, bad eyes, bad limbs, bad hands, rotating camera, blurry camera, shaking camera. Deformation, low-resolution, blurry, ugly, distortion."
sample_step_slider	The number of sampling steps for the diffusion model's denoising process. More steps can produce richer details but increase generation time.	int	30
cfg_scale_slider	The prompt guidance scale. Higher values increase prompt alignment but may reduce diversity.	float	6
sampler_dropdown	The sampling algorithm to use. Valid values: Euler, EulerA, DPM++, PNDM, DDIM	string	Euler
width_slider	The width of the generated video.	int	672
height_slider	The height of the generated video.	int	384
length_slider	The number of frames in the generated video.	int	144
is_image	Specifies whether the input is an image.	bool	False
lora_alpha_slider	The weight of the LoRA model parameters.	float	0.55
seed_textbox	The random seed.	int	43
lora_model_path	The path to an additional LoRA model. If specified, the LoRA model is loaded for the current request and removed after the request is complete.	string	none
base_model_path	The path of the transformer model to update.	string	none
motion_module_path	The path of the motion_module model to update.	string	none
generation_method	The generation type. Valid values: Video Generation, Image Generation.	string	none

Step 3: (Optional) Fine-tune the model

You can fine-tune the model on a custom dataset to generate videos with specific styles or content. Follow these steps:

Log on to the PAI console. In the left-side navigation pane, click Workspaces, then find and click the name of your target workspace.
In the left-side navigation pane, click Quick Start > Model Gallery.
In Model Gallery, search for EasyAnimate HD Long Video Generation Model and click Fine-tune to open the configuration page.
Set Source to Public Resources. For Instance type, select an instance with an A10 or higher GPU. Configure hyperparameters as needed and keep the default values for other parameters.

To fine-tune the model with a custom dataset, use the following guidance:
Custom dataset
1. Prepare a data folder and a meta file. The data folder contains the images and videos for training. The meta file is in JSON format, where each entry consists of a file path, text description, and data type, represented by the file_path, text, and type fields, respectively. For example:
```
[
    {
        "file_path": "00031-3640797216.png",
        "text": "1girl, black_hair",
        "type": "image"    },
    {
        "file_path": "00032-3838108680.png",
        "text": "1girl, black_hair",
        "type": "image"    }
]
```
  For video data, set type to video. For image data, set type to image.
2. On the training configuration page, click Select OSS File or Directory, and then upload and select the data folder and meta file.
Click Train > Confirm to create the training job. With the configuration in this tutorial, the training process takes about 40 minutes. When the job status changes to Successful, the model fine-tuning is complete.

To view training job details later, in the left-side navigation pane, click Model Gallery > Job Management > Training Jobs, and then click the job name.
In the upper-right corner, click Deploy to deploy the fine-tuned model. When the status changes to Running, the deployment is complete.
On the Service details page, click View WEB Application at the top of the page.

To view service details later, in the left-side navigation pane, click Model Gallery > Job Management > Deployment Jobs, and then click the Service name.
In the WebUI, select the trained LoRA model to generate videos. For API usage, see Step 2.

In the Model Checkpoints section, select the LoRA model from the Select LoRA model drop-down list, and set its weight by using the LoRA alpha slider.

Production recommendations

Stop instances or services to save costs: This tutorial creates DSW instances and EAS model services by using public resources. To avoid incurring further charges, stop or delete these instances and services when you no longer need them.
- On the Interactive Modeling (DSW) page, find the target instance. In the Actions column, click Stop to stop a running instance, or click Delete to delete a stopped instance.
- On the Model Gallery page, click the Task Management tab, and then click the Deployment Tasks sub-tab. Find the target service and click Stop or Delete in the Actions column.
Use EAS for production deployment: For production environments, use Solution 2 to deploy the model to EAS with a single click. If you used Solution 1, you can deploy the model to EAS by creating a custom image. For more information, see Deploy models as online services.

EAS provides the following features for production workloads:
- Use the stress testing feature of EAS to test the concurrency level that the service endpoint can support.
- Use the auto scaling feature of EAS to automatically scale instances to handle traffic spikes and dips, ensuring stable service operation.
- Use the log monitoring and alerting feature of EAS to track service health in real time and improve system stability and security.

Generate high-definition long videos using EasyAnimate