TokenWorks quick start

Prerequisites

Before you begin, make sure you have the following:

• An Alibaba Cloud account with real-name verification completed.

• PAI service activated, and a PAI workspace created.

Step 1: Create a workstation

A workstation is the central entry point for TokenWorks — an LLM intelligent routing service that schedules and manages all inference services deployed within it. Create a workstation before deploying any inference services.

1. Log in to the PAI console, and in the left navigation pane, choose Model Deployment > TokenWorks.

2. On the welcome page, click + Create Workstation.

3. Configure the following key parameters and leave the rest at their defaults (for full parameter descriptions, see Create and manage workstations):

   • Workstation Name: Enter a custom name, for example workstation_demo.

   • Resource Type: Select Public Resource Group.

   • VPC Configuration: Configure the VPC, vSwitch, and security group for the workstation.

   • Redis Configuration: When configured, the workstation stores user information and usage statistics in Alibaba Cloud Tair (Redis-compatible). Without this setting, data is stored in local memory only and is lost when the service stops. Configure this for production environments. Choose a configuration method based on whether you already have a Redis instance:

     • Create New Instance: Recommended for first-time users. The system automatically creates a Tair instance and configures the allowlist. After creation, the Redis address and password are displayed — save them securely.

     • Select Existing Instance: If you already have a Tair or Redis instance under your account, reuse it to avoid creating a duplicate and incurring extra charges.

     • Manual Input: Make sure the VPC of the Redis instance matches the VPC configured for the workstation, or the connection will fail.

4. Review the total cost at the bottom of the page, then click Create Workstation.

Step 2: Deploy an inference service

1. In the left navigation pane, click Service to open the inference services page.

2. Click + Create Service and configure the following parameters (this example deploys a ModelGallery model; for full service configuration options, see Create and manage inference services):

   • Service Name: Enter a custom name, for example test_demo.

   • Model Type: Select ModelGallery.

   • Model Selection: Select the target model and deployment template. For example, select Qwen3.5-4B as the model and SGLang-single node as the deployment template.

     The deployment template determines the inference engine, deployment method, and available resources. Different templates have different VRAM requirements and throughput capacities — refer to the template description to choose.

   • Resource Type: Select Public Resources.

   • Replicas: 1.

   • Deployment Resources: Select a resource specification that fits your workload (the template pre-filters compatible types), for example ecs.gn7i-c8g1.2xlarge.

   Important

   Before creating the inference service, deploy the KV Cache service. For multi-turn conversations, RAG, and similar scenarios, this reduces time-to-first-token (TTFT) and improves throughput. For details, see Configure KV Cache.

3. Review the cost at the bottom of the page, then click Deployment. The service starts in Creating status. When the status changes to Running, the service is ready to accept requests. This typically takes 3–5 minutes.

Step 3: Call the inference service

The workstation provides a unified entry point that routes all requests through intelligent load balancing and failover, and is the recommended way to call inference services.

In the upper-right corner of the page, click Call Information. The panel shows workstation-level credentials and ready-to-run call examples with pre-filled values, supporting Bash/Python and both streaming and non-streaming modes. Copy and run the examples to verify connectivity.

Two Base URL formats are available — choose based on your network environment:

• Public Network: Accessible from external networks.

• Internal Network: Available within the same VPC only, with lower latency.

To test a specific inference service independently, click Invoke in the service list, then use the Service Invocation panel to get that service's individual Base URL and API key. The call format is identical to the workstation entry point.

Step 4: Assign API keys to users

Assign each caller an individual API key in user management. The system automatically records token consumption and request counts per user, making it easy to track usage and allocate costs across team members.

1. In the left navigation pane, click User Management to open the user management page.

2. Click Create user, enter a username, and select a role. The system generates and displays an individual API key for that user. Click the copy icon to copy it.

3. In the examples below, replace <BASE_URL> with the Base URL from the workstation routing panel in Step 3, and replace <API_KEY> with the user-level API key from the previous step, then send the request.

   Note

   After calling with the user-level API key, the user's consumption record appears in the user ranking section of the overview in Step 5. Use a prompt that generates a long response so that real-time monitoring has observable data to display.

   curl

   curl -X POST "<BASE_URL>/v1/chat/completions" \
     -H "Authorization: Bearer <API_KEY>" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "Qwen3.5-4B",
       "messages": [
         {"role": "user", "content": "Explain the Transformer architecture in detail, covering self-attention, multi-head attention, positional encoding, and feed-forward networks. Also explain how KV Cache reduces redundant computation during large language model inference. Write at least 1,500 words."}
       ],
       "stream": false
     }'

   Python

   from openai import OpenAI
   
   client = OpenAI(
       api_key="<API_KEY>",
       base_url="<BASE_URL>/v1",
   )
   
   response = client.chat.completions.create(
       model="Qwen3.5-4B",
       messages=[
           {"role": "user", "content": "Explain the Transformer architecture in detail, covering self-attention, multi-head attention, positional encoding, and feed-forward networks. Also explain how KV Cache reduces redundant computation during large language model inference. Write at least 1,500 words."}
       ],
   )
   
   print(response.choices[0].message.content)

Step 5: View usage data

After completing Step 4, view call data on the Overview and Real-Time Monitoring pages.

• Overview: Displays aggregated historical data for reviewing total call volume and per-user consumption summaries.

• Real-Time Monitoring: Refreshes at per-second frequency, suited for monitoring current service health.