Deploy DeepSeek-V4 and DeepSeek-R1

Core components

The solution is built on the PAI platform and consists of the following core components:

• Model Gallery: the entry point for model distribution and deployment, providing preset DeepSeek models and deployment configurations.

• Elastic Algorithm Service (EAS): the core service that hosts model deployment and inference. It automatically manages underlying compute resources, such as GPUs, and launches model service instances.

• Inference acceleration engines (SGLang/vLLM/BladeLLM): optimize model inference performance.

  • SGLang/vLLM: provides interfaces that are fully compatible with the OpenAI API for migrating existing applications.

  • BladeLLM: an in-house high-performance inference framework that delivers better performance in specific scenarios.

• API gateway: provides a secure channel for accessing model services. You can call the services by using the service endpoint and token.

Deployment method

For large-scale models, in addition to single-node deployment, Model Gallery also offers one-click deployment solutions such as distributed deployment and EP+PD separation.

For example, DeepSeek-V4-Pro-FP8 and DeepSeek-V4-Flash-FP8 can both be deployed by using the EP+PD separation method.

Step 1: Deploy the model service

1. Log on to the PAI console and select the target region in the upper-left corner. From the left-side navigation pane, go to Workspaces and select the target workspace.

2. In the workspace, navigate to QuickStart > Model Gallery.

3. Search for and select the target model in the model list, such as DeepSeek-R1-Distill-Qwen-7B, and click the model card to go to the details page.

4. Click Deploy in the upper-right corner of the page and configure the following parameters.

   • Deployment Template: By default, the card type is A10, the inference engine is SGLang, the deployment method is single-node, and the deployment scenario is balanced.

   • Deployment Resource: Select public resources or dedicated resources, and select an appropriate GPU specification based on the model requirements.

     • By default, public resources are used and a recommended specification is provided. If no specifications are available, try switching to another region.

       Important

       When you deploy by using public resources, billing starts as soon as the instance is provisioned, and charges are based on the usage duration, even if there are no calls. Stop the service promptly after testing.

     • If you select Resource Quota, make sure to choose a matching inference engine and deployment template based on the instance type. For example, if you use Zhenwu PPU, you can select SGLang as the inference engine and PPU-810E as the card type in the deployment template.

     An example of a recommended GPU specification is ecs.gn7i-c16g1.4xlarge (16 vCPUs, 60 GiB, 1 × NVIDIA A10), which costs approximately CNY 11.1 per hour.

5. After you confirm that all configurations are correct, click Deploy. The system begins to create the service.

   Note

   For a large model, such as the full-version DeepSeek-R1, the model loading process might take 20 to 30 minutes.

6. You can view the status of the deployment job on the Model Gallery > Job Management > Deployment Jobs page. Click a service name to go to the service details page. You can also click More > More Information in the upper-right corner to go to the model service details page in PAI-EAS for more information.

   On the service details page, you can click View Deployment Events next to the Status field for more details, and click View Call Information in the Call Information field to obtain the service endpoint and token. You can also choose Elastic Algorithm Service (EAS) from the left-side navigation pane to go to the EAS service management page.

Step 2: Online debugging

On the Model Gallery > Job Management > Deployment Jobs page, click the name of the deployed service and switch to the Online Debugging tab, which supports both Conversation Debugging and API Debugging.

If only the API debugging page is available, the following example shows how to debug the chat API of a large language model (LLM) service:

1. Set the request method and path: On the debugging tab, select POST as the HTTP method and append /v1/chat/completions to the address to form the complete request URL <EAS_ENDPOINT>/v1/chat/completions. In the URL, <EAS_ENDPOINT> is the service endpoint, which is usually pre-populated. The standard format is http://<uid>.<region-id>.pai-eas.aliyuncs.com/api/predict/<service-name>.

   For services deployed with SGLang/vLLM, you can find more supported APIs at <EAS_ENDPOINT>/openapi.json.

2. Construct the request body.

   If the prompt is "What is 3 + 5?", the request body is formatted as follows.

   The value of the model parameter is the model name obtained from the model list API <EAS_ENDPOINT>/v1/models. This example uses DeepSeek-R1-Distill-Qwen-7B.

   In the Body section, select raw and enter the following JSON request body:

   {
       "model": "DeepSeek-R1-Distill-Qwen-7B",
       "messages": [
           {
               "role": "user",
               "content": "What is 3 + 5?"
           }
       ]
   }

3. Send the request.

   Click Send Request. The Response section on the right shows the status code 200, and the Body section displays the model response in JSON format.

{
    "messages": [
        {
            "role": "user",
            "content": "What is 3 + 5?"
        }
    ],
    "max_tokens": 2000
}

The model you select determines the required compute resources and deployment costs. DeepSeek models are divided into full-version models and distilled version models, which differ significantly in resource requirements.

• Development and testing: We recommend that you use a distilled version model, such as DeepSeek-R1-Distill-Qwen-7B. These models have low resource requirements (typically a single GPU with 24 GB of GPU memory), can be deployed quickly at a low cost, and are suitable for rapid feature validation.

• Production environments: Evaluate both model performance and cost. DeepSeek-R1-Distill-Qwen-32B strikes a good balance between performance and cost. If you need stronger model performance, you can choose a full-version model, but full-version models require multiple high-end GPUs (such as 8 GPUs with 96 GB of GPU memory each) and significantly increase costs.

The following table lists the minimum configurations required for different model versions and the maximum number of tokens supported when different instance types and inference engines are used.

Cost breakdown

For services that use public resources, billing is calculated by the minute, starting from when an instance is provisioned until it is stopped or deleted. Bills are settled hourly. Charges accrue even when the service is idle. Stopping the service halts billing.

For more information, see Elastic Algorithm Service (EAS) billing.

Cost control

• Clean up promptly: After development and testing, immediately stop or delete the service to control costs.

• Use trial resources: If you are using EAS for the first time, go to Alibaba Cloud Free Tier to claim PAI-EAS trial resources. After you claim the resources, you can choose to deploy a model with a minimum configuration of A10, such as DeepSeek-R1-Distill-Qwen-7B, and change the resource specification to an instance type provided in the trial during deployment.

• Select an appropriate model: In non-production environments, prioritize lower-cost distilled version models.

• Use preemptible resources: For non-production tasks, you can enable the preemptible mode during deployment. Note that certain conditions must be met for a successful bid, and there is a risk of resource instability.

• Long-term usage discounts: For long-running production services, you can reduce costs by purchasing a savings plan or prepaid resources.

Key risks

• Unexpected costs: Forgetting to stop the service results in continuous billing. Always clean up resources immediately after use.

• BladeLLM output truncation: When you use the BladeLLM engine, if max_tokens is not specified in the API request, the output is truncated to 16 tokens, which might cause the feature to not work as expected.

• Incorrect API usage:

  • When you call a DeepSeek-R1 series model, including a system prompt in messages might cause unexpected behavior.

  • The API request URL must end with a path such as /v1/chat/completions. Otherwise, a 404 error is returned.

• Resource inventory: When you deploy large models (especially full-version models), the inventory of high-end GPU resources in specific regions might be tight, causing deployment failures or long waiting times. You can try switching to another region.

Choosing an inference engine

Service stuck waiting

Possible causes:

• Insufficient machine resources in the current region.

• Model loading takes a long time due to the large model size (20 to 30 minutes for models such as DeepSeek-R1 and DeepSeek-V3).

Wait patiently for a while. If the service still fails to start as expected, we recommend that you try the following steps:

1. Go to Job Management-Deployment Jobs to open the deployment job details page. Click More > More Information in the upper-right corner of the page to go to the model service details page in PAI-EAS and view the service instance status.

   If the Instance Status column shows Insufficient Inventory, the instance cannot be scheduled because of insufficient resources.

2. Stop the current service, switch to another region in the upper-left corner of the console, and deploy the service again.

   Note

   Models with an extremely large number of parameters, such as DeepSeek-R1 and DeepSeek-V3, require 8 GPUs to start the service (resource inventory is tight). You can choose to deploy smaller distilled models such as DeepSeek-R1-Distill-Qwen-7B (resource inventory is abundant).

API call returns 404

Check whether an OpenAI API suffix, such as v1/chat/completions, has been added to the URL. For more information, see the API call instructions on the model introduction page.

If you use a vLLM accelerated deployment, check that the model parameter in the request body of the chat API is set to the correct model name. You can obtain the model name from v1/models.

Request timeout

The default gateway request timeout for deployments is 180 seconds. To extend it, you can configure a dedicated gateway and submit a ticket to adjust the request timeout of the dedicated gateway. The timeout can be extended up to 600 seconds.

No "web search" feature

The "web search" feature is not enabled by deploying the model service alone; it requires building a separate AI application (agent) on top of the service.

You can use LangStudio, the LLM application development platform of PAI, to build a web search AI application. For more information, see Build a DeepSeek Web Search Application with LangStudio.

Model skips thinking

If the DeepSeek-R1 model sometimes skips the thinking process, use the updated chat template from DeepSeek that forces thinking:

1. Modify the startup command.

   In the service configuration, edit the JSON configuration and modify the containers-script field to add "--chat-template /model_dir/template_force_thinking.jinja" (you can add it after "--served-model-name DeepSeek-R1").

   For an already deployed service, go to Model Gallery > Job Management > Deployment Jobs, click the name of the deployed service, and then click Update service in the upper-right corner of the details page to go to the configuration page.

2. Modify the request body. In each request, add {"role": "assistant", "content": "<think>\n"} at the end of the message.

Disabling thinking mode

DeepSeek-R1 series models do not currently support disabling the thinking process.

Multi-turn conversations

The model service does not save conversation history. The client must store the conversation history and add it to the requests sent to the model. The following example shows a service deployed with SGLang.

curl -X POST \
    -H "Content-Type: application/json" \
    -H "Authorization: <EAS_TOKEN>" \
    -d '{
        "model": "<model_name>",
        "messages": [
         {
            "role": "user", 
            "content": "Hello"
        },
        {
            "role": "assistant",
            "content": "Hello! Nice to meet you. How can I help you?"
        },
        {
            "role": "user",
            "content": "What was my previous question?"
        }
        ]
    }' \
    <EAS_ENDPOINT>/v1/chat/completions