Deploy DeepSeek-V4 and DeepSeek-R1

Core components

This solution is built on PAI and includes the following core components:

• Model Gallery: Serves as the entry point for model distribution and deployment. It provides pre-configured DeepSeek models and their corresponding deployment configurations.

• Elastic Algorithm Service (EAS): The core service that hosts model deployment and inference. It automatically manages underlying compute resources, such as GPUs, and starts model service instances.

• Inference acceleration engine (SGLang/vLLM/BladeLLM): Optimizes model inference performance.

  • SGLang/vLLM: Provides interfaces that are fully compatible with the OpenAI API, which simplifies migrating existing applications.

  • BladeLLM: A proprietary high-performance inference framework that provides superior inference performance in specific scenarios.

• API gateway: Provides a secure channel to access the model service. It supports calling the model service with a service endpoint and an authentication token.

Deployment method

For large-scale models, in addition to single-node deployment, Model Gallery also offers one-click deployment solutions such as distributed deployment and EP+PD separation.

For example, DeepSeek-V4-Pro-FP8 and DeepSeek-V4-Flash-FP8 can both be deployed by using the EP+PD separation method.

Step 1: Deploy the model service

1. Log on to the PAI console and select the target region in the upper-left corner. From the navigation pane on the left, go to Workspaces and select the target workspace.

2. In the workspace, go to QuickStart > Model Gallery.

3. In the model list, search for and select the target model, such as DeepSeek-R1-Distill-Qwen-7B, to open the model details page.

4. Click Deploy in the upper-right corner, and then configure the following parameters.

   • Inference Engine: SGLang or vLLM is recommended.

   • Deployment Resource: Choose a public resource or dedicated resource, and select the appropriate GPU specification based on the model's requirements.

     • By default, a public resource is used, and a recommended specification is provided. If no specifications are available, try switching the region.

       Important

       When you deploy by using a public resource, billing starts as soon as the service instance provisions the resource, with charges based on duration, even if there are no calls. Stop the service promptly after testing.

     • If you select Resource Quota, make sure to choose the corresponding inference engine and deployment template for your instance type. For example, if you use a GP7V instance type, you can select SGLang for the Inference Engine and must select Single-Node-GP7V for the Deployment Template.

   The Deployment Template is set to Single-Node by default. An example of a recommended GPU specification is ecs.gn7i-c16g1.4xlarge (16 vCPUs, 60 GiB, 1 × NVIDIA A10), which costs approximately CNY 11.1 per hour.

5. After confirming that all configurations are correct, click Deploy. The system begins creating the service.

   Note

   For a large model, such as the full-version DeepSeek-R1, the model loading process might take 20 to 30 minutes.

6. You can view the status of the deployment job on the Model Gallery > Job Management > Deployment Jobs page. Click the service name to open the service details page. You can also click More Information in the upper-right corner to view the model service details page in EAS.

   On the service details page, click View Deployment Events next to the Status field for more details. To obtain the service endpoint and token, click View Call Information in the Call Information section.

Step 2: Online debugging

On the Model Gallery > Job Management > Deployment Jobs page, click the name of the deployed service and switch to the Online Debugging tab, which supports both Conversation Debugging and API Debugging.

If only the API debugging page is available, the following steps provide an example using the chat API of a large language model (LLM) service:

1. Confirm the request path: <EAS_ENDPOINT>/v1/chat/completions. In this path, <EAS_ENDPOINT> is the service endpoint, which is usually pre-filled.

   For services deployed with SGLang/vLLM, you can retrieve more supported APIs from <EAS_ENDPOINT>/openapi.json.

2. Construct the request body.

   If the prompt is "What is 3 + 5?", the request body is formatted as follows.

   The value for the model parameter is the model name obtained from the model list API <EAS_ENDPOINT>/v1/models. This example uses DeepSeek-R1-Distill-Qwen-7B.

   {
       "model": "DeepSeek-R1-Distill-Qwen-7B",
       "messages": [
           {
               "role": "user",
               "content": "What is 3 + 5?"
           }
       ]
   }

3. Send the request.

   On the Online Debugging tab, select POST as the HTTP method and enter the service endpoint in the URL field, for example, http://<service_address>/v1/chat/completions. In the Body section, select raw and enter the JSON request body, including the model, such as DeepSeek-R1-Distill-Qwen-7B, and messages fields. Click Send Request. The Response area on the right displays a status code of 200, and the model's response is shown in the body in JSON format.

{
    "messages": [
        {
            "role": "user",
            "content": "What is 3 + 5?"
        }
    ],
    "max_tokens": 2000
}

Your choice of model determines the required compute resources and deployment costs. DeepSeek models are available as "full-version" and "distilled version" variants, which have vastly different resource requirements.

• Development and testing: We recommend that you use a distilled version model, such as DeepSeek-R1-Distill-Qwen-7B. These models have a smaller resource footprint, typically a single GPU with 24 GB of GPU memory, deploy quickly, and are cost-effective, which makes them ideal for rapid feature validation.

• Production environment: Evaluate based on a balance of performance and cost. The DeepSeek-R1-Distill-Qwen-32B model strikes a good balance between effectiveness and cost. If you require higher model performance, choose a full-version model. This requires multiple high-end GPUs, such as eight GPUs with 96 GB of GPU memory each, which significantly increases costs.

The following table lists the minimum configurations for different model versions and the maximum number of tokens supported by different instance types and inference engines.

Cost breakdown

For services that use public resources, billing is calculated by the minute, starting from when an instance is provisioned until it is stopped or deleted. Bills are settled hourly. Charges accrue even when the service is idle. Stopping the service halts billing.

For more information, see Billing of Elastic Algorithm Service (EAS).

Cost control

• Clean up promptly: After development and testing, immediately stop or delete the service to effectively control costs.

• Use trial resources: If you are using EAS for the first time, you can go to Alibaba Cloud Free Tier to claim PAI-EAS trial resources. After you claim the resources, you can choose to deploy a model with a minimum configuration of A10, such as DeepSeek-R1-Distill-Qwen-7B, and modify the resource specification to the instance type provided in the trial event during deployment.

• Select an appropriate model: In non-production environments, prioritize lower-cost distilled version models.

• Use preemptible instances: For non-production tasks, you can enable the preemptible mode during deployment. Note that certain conditions must be met for a successful bid, and there is a risk of resource instability.

• Long-term usage discounts: For long-running production services, you can reduce costs by purchasing a savings plan or prepaid resources.

Key risks

• Unexpected costs: Forgetting to stop the service results in continuous billing. Always clean up resources immediately after use.

• BladeLLM output truncation: When you use the BladeLLM engine, if the max_tokens parameter is not specified in the API request, the output is truncated to 16 tokens, which might prevent the feature from working as expected.

• Incorrect API usage:

  • When you call a DeepSeek-R1 series model, including a system prompt in the messages might cause unexpected behavior.

  • The API request URL must end with a path such as /v1/chat/completions. Otherwise, a 404 error is returned.

• Resource inventory: Limited inventory of high-end GPU resources in a specific region can lead to deployment failures or long waiting times. You can try switching to another region.

Choosing an inference engine

Service stuck waiting

Possible reasons:

• Insufficient machine resources in the current region.

• The model is large, and model loading takes a long time. For large models such as DeepSeek-R1 and DeepSeek-V3, this can take 20 to 30 minutes.

You can wait for a period of time. If the service still fails to start after an extended period, we recommend that you try the following steps:

1. Go to the Job Management > Deployment Jobs page to view the deployment job details. In the upper-right corner, click More > More Information to go to the PAI-EAS model service details page and check the service instance status.

   If the Instance Status column shows Insufficient Inventory, it means the instance cannot be scheduled due to a lack of resources.

2. Stop the current service, and then switch to another region in the upper-left corner of the console to redeploy the service.

   Note

   For very large parameter models such as DeepSeek-R1 and DeepSeek-V3, 8 GPUs are required to start the service (resource inventory is tight). You can choose to deploy smaller, distilled models such as DeepSeek-R1-Distill-Qwen-7B (resource inventory is abundant).

API call returns 404

Check whether an OpenAI API suffix, such as v1/chat/completions, has been added to the URL. For more information, refer to the API call instructions on the model's introduction page.

If you are using a vLLM-accelerated deployment, check that the model parameter in the request body of the conversation API is set to the correct model name. You can obtain the model name from v1/models.

Request timeout

The default gateway request timeout is 180 seconds. To extend it, configure a Dedicated Gateway and submit a ticket to adjust the timeout. The maximum is 600 seconds.

No "web search" feature

The "web search" feature is not enabled by deploying the model service alone; it requires building a separate AI application (Agent) on top of the service.

Use LangStudio, PAI's large model application development platform, to build a web search AI application. For more information, see Build a DeepSeek-based web search application flow using LangStudio and Alibaba Cloud Information Query Service.

Model skips thinking

If the DeepSeek-R1 model sometimes skips the thinking process, use the updated chat template from DeepSeek that forces thinking:

1. Modify the startup command.

   In the service configuration, edit the JSON configuration. Modify the containers-script field to add --chat-template /model_dir/template_force_thinking.jinja, which can be added after --served-model-name DeepSeek-R1.

   For an already deployed service, go to Model Gallery > Job Management > Deployment Jobs, click the deployed service name, and then click Update service in the upper-right corner to go to the configuration page.

2. Modify the request body. In each request, add {"role": "assistant", "content": "<think>\n"} at the end of the message.

Disabling thinking mode

DeepSeek-R1 series models do not support disabling the thinking process.

Multi-turn conversations

The model service does not save conversation history. The client application must store the history and include it in subsequent requests. The following example shows a multi-turn conversation with a service deployed by using SGLang.

curl -X POST \
    -H "Content-Type: application/json" \
    -H "Authorization: <EAS_TOKEN>" \
    -d '{
        "model": "<model_name>",
        "messages": [
         {
            "role": "user", 
            "content": "Hello"
        },
        {
            "role": "assistant",
            "content": "Hello! I''m glad to see you. What can I help you with?"
        },
        {
            "role": "user",
            "content": "What was my previous question?"
        }
        ]
    }' \
    <EAS_ENDPOINT>/v1/chat/completions