Use Function Compute to deploy the Qwen LLM for AI chat

Background information

The Qwen model takes text instructions (prompts) and multi-turn conversation history as input, and generates responses based on this information. During this process, the text is converted into a sequence of tokens that the language model can process. A token is the basic unit that the model uses to represent natural language text. You can think of it as a character or a word. For Chinese text, one token usually corresponds to one character. For English text, one token usually corresponds to three to four letters or one word. For example, the English text "Nice to meet you." is converted to the sequence ['Nice', ' to', ' meet', ' you', '.'].

Because the computational cost of model invocations depends on token sequence length, a greater number of input or output tokens increases the model’s computation time. We bill based on the number of input and output tokens. You can find the number of tokens used in each invocation from the usage field in the API response. You can also use the Token Calculator or call the Token Calculation API to estimate the number of tokens in your text.

For more information about the Qwen model series, including pricing, see Model List.

Prerequisites

Obtain an API key

Deployment instructions

In this topic, Step 1 and Step 2 describe how to deploy an application using the Function Compute console to create a function, write code, and install the SDK. For convenience, you can also click the start-qwen-api-messages or start-qwen-api-prompt button and follow the prompts to complete one-click deployment of the application. After deployment, click the application to go to its product page, locate the function resources, and then directly test the results according to Step 3.

Step 1: Create a web function

1. Log on to the Function Compute console. In the left-side navigation pane, click Functions.

2. In the top navigation bar, select a region. On the Functions page, click Create Function.

3. On the Create Function page, select Web Function. Configure the following key parameters and then click Create.

   • Instance Concurrency: 100

   • Runtime: Custom Runtimes/Python/Python 3.10

   • Code Upload Method: Use Sample Code

   • Function Role: AliyunFcServerlessDevsRole

4. On the Function Details page, click the Configuration tab, locate Advanced Settings, and click Modify to its right. In the Advanced Configuration panel, find the Environment Variables section, set the value of the DASHSCOPE_API_KEY variable to the API-KEY you obtained from the Alibaba Cloud Model Studio console, and then click Deploy.

Step 2: Write and deploy the Qwen AI chat code

After the function is created, you can start writing the AI chat code that uses Qwen.

1. On the function details page, click the Code tab and write the code in the code editor.

   The following Python code shows how to call the Qwen model to respond to a user instruction.

   Call with messages

   import dashscope
   import random
   
   from flask import abort
   from flask import Flask
   from flask import request
   from http import HTTPStatus
   
   app = Flask(__name__)
   
   
   def call_with_messages(content):
       messages = [{'role': 'system', 'content': 'You are a helpful assistant.'},
                   {'role': 'user', 'content': content}]
       response = dashscope.Generation.call(
           dashscope.Generation.Models.qwen_turbo,
           messages=messages,
           # set the random seed, optional, default to 1234 if not set
           seed=random.randint(1, 10000),
           result_format='message',  # set the result to be "message" format.
       )
       if response.status_code == HTTPStatus.OK:
           print(response)
       else:
           print('Request id: %s, Status code: %s, error code: %s, error message: %s' % (
               response.request_id, response.status_code,
               response.code, response.message
           ))
       return response
   
   
   @app.route("/invoke", methods=['POST', 'GET'])
   def index():
       bytes = request.stream.read()
       if bytes:
           payload = str(bytes, encoding='utf-8')
           print("Request Payload: " + payload)
           return call_with_messages(payload)
       else:
           abort(403)
   
         
   if __name__ == "__main__":
       app.run(host="0.0.0.0", port=9000)

   Call with a prompt

   import dashscope
   import random
   
   from flask import abort
   from flask import Flask
   from flask import request
   from http import HTTPStatus
   
   app = Flask(__name__)
   
   
   def call_with_prompt(prompt):
       response = dashscope.Generation.call(
           model=dashscope.Generation.Models.qwen_turbo,
           prompt=prompt
       )
       # The response status_code is HTTPStatus.OK indicate success,
       # otherwise indicate request is failed, you can get error code
       # and message from code and message.
       if response.status_code == HTTPStatus.OK:
           print(response.output)  # The output text
           print(response.usage)  # The usage information
       else:
           print(response.code)  # The error code.
           print(response.message)  # The error message.
       return response
   
   
   @app.route("/invoke", methods=['POST', 'GET'])
   def index():
       bytes = request.stream.read()
       if bytes:
           payload = str(bytes, encoding='utf-8')
           print("Request Payload: " + payload)
           return call_with_prompt(payload)
       else:
           abort(403)
   
      
   if __name__ == "__main__":
       app.run(host="0.0.0.0", port=9000)

2. In the terminal of the WebIDE, run pip3 install dashscope -t . to install the DashScope SDK. After the installation is complete, click Deploy.

Step 3: Test the function

On the function details page, click the Test tab. Enter an event name and the event content, and then click Test Function. After the execution is complete, you can view the results.

This topic uses "How do I make tomato and beef brisket stew?" as an example of event content. The returned result of the function test is as follows:

{
    "code": "",
    "message": "",
    "output": {
        "choices": [
            {
                "finish_reason": "stop",
                "message": {
                    "content": "Ingredients:\n500 g beef brisket, 2 tomatoes, 1 green onion, 3 slices of ginger, 2 cloves of garlic, 1 star anise, 2 bay leaves, 4 dried chilies, 1 tablespoon cooking wine, 1 tablespoon light soy sauce, 1 tablespoon dark soy sauce, 1 teaspoon sugar, salt to taste, water as needed.\n\nInstructions:\n1. Cut the beef brisket into chunks. Place them in a pot with cold water and cooking wine. Bring to a boil, skim off the foam, then remove the brisket and drain.\n2. Heat oil in a wok. Sauté the green onion, ginger, garlic, star anise, bay leaves, and dried chilies until fragrant.\n3. Add the tomatoes and stir-fry until they are soft and release their juice.\n4. Add the beef brisket chunks and continue to stir-fry evenly. Season with light soy sauce, dark soy sauce, and sugar.\n5. Pour in enough water to cover the ingredients. Bring to a boil over high heat, then reduce to a low-medium heat and simmer for about 1 hour until the beef is tender and the sauce has thickened.\n6. Finally, add salt to taste and serve.",
                    "role": "assistant"
                }
            }
        ],
        "finish_reason": null,
        "text": null
    },
    "request_id": "61dbd6b0-810b-9115-ad26-65e9ba93f84d",
    "status_code": 200,
    "usage": {
        "input_tokens": 15,
        "output_tokens": 212,
        "total_tokens": 227
    }
}

{
    "code": "",
    "message": "",
    "output": {
        "choices": null,
        "finish_reason": "stop",
        "text": "\n\nHere are the steps to make tomato and beef brisket stew:\n\nIngredients:\n- 500 g beef brisket\n- 3 tomatoes\n- Ginger slices, to taste\n- Green onion sections, to taste\n- Cooking wine, to taste\n- Light soy sauce, to taste\n- Brown sugar, to taste\n- 2 star anise\n- 1 small piece of cinnamon bark\n- Salt, to taste\n- Water, as needed\n\nSteps:\n\n1. Cut the beef brisket into chunks, blanch in boiling water, then remove, rinse, and set aside.\n\n2. Heat oil in a wok. Sauté the ginger slices and green onion sections until fragrant.\n\n3. Add the beef brisket and pan-fry until lightly browned on both sides.\n\n4. Add cooking wine, light soy sauce, brown sugar, star anise, and cinnamon bark, and continue to pan-fry.\n\n5. Add enough water to cover the beef brisket. Bring to a boil over high heat and skim off any foam.\n\n6. Reduce the heat to low-medium and simmer for about 1 hour until the beef is tender.\n\n7. During the simmering process, you can adjust the seasoning, such as adding salt.\n\n8. Cut the tomatoes into chunks and add them to the stew. Cook for about another 10 minutes.\n\n9. Before serving, you can garnish with some chopped green onions.\n\nThese are the detailed steps for making tomato and beef brisket stew. Hope this helps!"
    },
    "request_id": "a9a5f430-745d-9f5d-b406-8588978f638b",
    "status_code": 200,
    "usage": {
        "input_tokens": 8,
        "output_tokens": 278,
        "total_tokens": 286
    }
}

After testing, if you no longer need this application, delete it and its associated resources promptly to avoid incurring further charges.

References

• Qwen API details: Use Qwen through an API.