Deploy Text Generation Model API-Alibaba Cloud Model Studio(Model Studio)-阿里云帮助中心

Deploy a trained text model as an online API service.

Prerequisites

Supported region: The model deployment API is currently only available in the China (Beijing) region. If you are using a different region, please complete the model deployment through the Model Studio console for that region.
Account permissions: If using an Alibaba Cloud sub-account (RAM user), you need to grant the sub-account permissions for model invocation, training, and deployment.
Configure environment variables: You have successfully obtained an API Key and configured it as an environment variable.
Read deployment documentation: It is recommended to first read Model deployment overview and Create a model deployment using API to understand the usage and basic steps of model deployment.

Create a deployment

China (Beijing)

POST https://dashscope.aliyuncs.com/api/v1/deployments

For Windows CMD, replace $DASHSCOPE_API_KEY with %DASHSCOPE_API_KEY%. For PowerShell, replace it with $env:DASHSCOPE_API_KEY

Request parameters

Headers

Content-Type string (Required)

Fixed value: application/json

Authorization string (Required)

API Key authentication, in the format Bearer sk-xxxx.

Request Body

model_name string (Required)

The name of the model to deploy, corresponding to the Model ID in My Models. You can also obtain it from the output of the Create a tuning job API.

plan string (Required)

Deployment plan. Valid values:

mu: Model unit billing.
lora: LoRA shared deployment (token-based billing).
ptu: Provisioned throughput billing.

deploy_spec string (Conditionally required)

Deployment template. Required when plan is mu. Example: "deploy_spec": "MU1".

You can obtain it from the template_id field returned by the List deployable models API.

capacity integer (Conditionally required)

The number of resource units for the deployment, which must be an integer multiple of base_capacity. Different deploy_spec values have different constraints. For example, MU2 must be a multiple of 8, while MU5 can be 1.

billing_method string (Conditionally required)

Billing method. Required when plan is mu. Currently supports "POST_PAY" (pay-as-you-go).

enable_thinking boolean (Optional)

Only configurable when plan is mu. Supported by some models. Can be set to true or false.

max_context_length number (Optional)

Only configurable when plan is mu. Supported by some models. Example: "max_context_length": 131072.

rpm_limit number (Optional)

Only configurable when plan is mu. Supported by some models. Requests per minute.

tpm_limit number (Optional)

Only configurable when plan is mu. Supported by some models. Tokens per minute.

ptu_capacity object (Optional)

Only takes effect when plan is ptu. If not provided, defaults to 10,000 input_tpm and 1,000 output_tpm.

ptu_capacity properties

input_tpm number

The maximum input tokens per minute supported by the deployed model.

output_tpm number

The maximum output tokens per minute supported by the deployed model.

thinking_output_tpm number

Supported by some models. The maximum thinking output tokens per minute supported by the deployed model.

name string (Optional)

The display name of the model in the console. If not provided, the value of model_name will be used as the deployment name.

suffix string (Optional)

After model deployment, a new model name will be generated. The suffix is used to specify the suffix of the new model name, with a maximum length of 8 characters and must be globally unique. When deploying a model for the first time, the suffix can be omitted. For subsequent deployments of the same model, a suffix must be specified to distinguish between them.

Model unit billing

Billing is based on the usage duration of model units. Suitable for large-scale inference after model fine-tuning, with dedicated resources and flexible performance and cost control.

After executing the deployment command below, billing will start once the deployment is successful, even if you have not yet invoked the model. Please confirm the billing rules before executing the deployment command.

curl "https://dashscope.aliyuncs.com/api/v1/deployments" \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "name": "my_qwen_plus",
    "model_name": "qwen-plus-2025-12-01",
    "plan": "mu",
    "deploy_spec": "MU1",
    "enable_thinking": true,
    "capacity": 4,
    "billing_method": "POST_PAY",
    "max_context_length": 10000,
    "rpm_limit": 500,
    "tpm_limit": 1000
}'

Token-based billing

Billing is based on token usage. Suitable for cost-effective scenarios with lower concurrency and latency requirements. This mode offers the best pricing, with throughput/concurrency and generation speed preset by the platform and not user-adjustable.

curl "https://dashscope.aliyuncs.com/api/v1/deployments" \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model_name": "qwen3-8b-ft-202511132025-0260",
    "plan": "lora",
    "capacity": 1,
    "name": "qwen3-8b-ft"
}'

Provisioned throughput billing

Billing is based on the usage duration of provisioned throughput. Suitable for scenarios requiring stable throughput guarantees with high concurrency and low latency, and predictable traffic. Throughput/concurrency and generation speed are preset by the platform and not user-adjustable.

After executing the deployment command below, billing will start once the deployment is successful, even if you have not yet invoked the model. Please confirm the billing rules before executing the deployment command.

curl "https://dashscope.aliyuncs.com/api/v1/deployments" \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "name": "my_qwen_flash",
    "model_name": "qwen-flash-2025-07-28",
    "plan": "ptu",
    "ptu_capacity": {
        "input_tpm": 10000,
        "output_tpm": 1000
    }
}'

Deployment troubleshooting and performance tuning

max-num-seqs parameter is not configurable

Under both provisioned throughput billing and model unit billing plans, throughput, concurrency, and generation speed are all preset by the platform. Direct adjustment of inference engine parameters such as max-num-seqs in vLLM is not supported, and this parameter does not exist in the request parameters of this API.

Under the model unit billing plan, you can indirectly control throughput by selecting the model unit type (corresponding to the request parameter deploy_spec) and adjusting the number of deployment replicas (corresponding to the request parameter capacity).

Deployment template and resource isolation

Currently, the deployment template only supports single-machine deployment (Single-machine deployment - Enhanced general inference) and does not support multi-GPU instance isolation. For resource isolation, use the model unit billing plan, which provides dedicated computing resources for your workload.

Throttling error handling

When concurrent requests exceed the throttling threshold, the API returns HTTP 429 with error code Throttling.RateQuota and error message Requests rate limit exceeded, please try again later. You can handle this as follows:

Provisioned throughput billing: Adjust ptu_capacity values for input_tpm and output_tpm (corresponding to Input kTPM and Output kTPM in the console), or reduce the request frequency. Under this plan, the overflow strategy can be set to Auto overflow (switch to pay-as-you-go billing) or Use PTU capacity only (requests exceeding capacity will directly return 429).
Model unit billing: Adjust rpm_limit and tpm_limit, or reduce the request frequency.

Context length tuning

Under the model unit billing plan, you can configure the maximum context length via the request parameter max_context_length, with a value range of 1 to 262144 (the actual upper limit depends on the capabilities of the deployed model). This parameter limits the context size per request, thereby limiting the memory usage per request and reducing the risk of OOM errors.

When processing tasks with a large number of images, set this parameter based on the image dimensions and the number of images per request.

Response parameters

request_id string

The unique identifier of the request.

output object

Task details.

Properties

deployed_model string

The unique identifier of the deployed model. Used to query deployment status and invoke the model.

model_name string

The model identifier name.

status string

Deployment status:

PENDING: The deployment task is being created.
UPDATING: The deployment task is being updated.
RUNNING: The deployment task is running. The deployed model can process requests normally.
STOPPED: The deployment task has been stopped and will not incur charges.
DELETING: The deployment task is being deleted.
FAILED: The deployment task creation or update has failed.

base_model string

The base model used.

gmt_create string

The creation time of the deployment task.

gmt_modified string

The last modification time of the deployment task.

workspace_id string

The workspace ID associated with the Alibaba Cloud Model Studio API Key. See Get the Workspace ID.

charge_type string

Billing mode. post_paid indicates pay-as-you-go.

creator string

The Alibaba Cloud account ID of the creator.

modifier string

The Alibaba Cloud account ID of the last modifier.

plan string

The deployment plan.

base_capacity number

The minimum number of resource units required to run the base model.

ready_capacity number

The number of resource units that are ready and can immediately process requests.

model_unit_spec string

The model unit specification. Only returned when plan is mu.

enable_thinking boolean

Whether thinking mode is enabled. Only returned when plan is mu.

max_context_length number

The maximum context length limit. Only returned when plan is mu.

rpm_limit number

Requests per minute. Only returned when plan is mu.

tpm_limit number

Tokens per minute. Only returned when plan is mu.

ptu_capacity object

Provisioned throughput configuration. Only returned when plan is ptu.

ptu_capacity properties

input_tpm number

The maximum input tokens per minute supported by the deployed model.

output_tpm number

The maximum output tokens per minute supported by the deployed model.

thinking_output_tpm number

Supported by some models. The maximum thinking output tokens per minute supported by the deployed model.

code string

Error code. Returned when the call fails.

message string

Detailed error description. Returned when the call fails.

Success response example

Key fields: output.deployed_model (unique identifier of the deployed model) and output.status (deployment status).

{
    "request_id": "f2ae64f7-83cc-410c-bc0b-840443f7eb86",
    "output": {
        "deployed_model": "qwen-plus-2025-12-01-mu-xxxx",
        "gmt_create": "2025-06-17T11:00:38.68",
        "gmt_modified": "2025-06-17T11:00:38.68",
        "status": "PENDING",
        "model_name": "qwen-plus-2025-12-01",
        "base_model": "qwen-plus",
        "model_unit_spec": "MU1",
        "enable_thinking": true,
        "max_context_length": 10000,
        "rpm_limit": 500,
        "tpm_limit": 1000,
        "base_capacity": 1,
        "ready_capacity": 0,
        "workspace_id": "llm-v71tlv3d***",
        "charge_type": "post_paid",
        "creator": "175805416***",
        "modifier": "175805416***",
        "plan": "mu"
    }
}

Error response example

{
    "code": "InvalidParameter",
    "request_id": "BE213CDD-8A5C-59EE-9A67-055EAB0CB59B",
    "message": "The model xxx does not exist or is not deployable"
}

Next steps

Deployment is an asynchronous operation. After calling this API, you can use the Get deployment details API to query the deployment status.