Use large language models from Model Studio

Prerequisites

• You have activated Alibaba Cloud Model Studio and obtained an API key. For more information, see Obtain and configure an API key.

• You have imported the API key. For more information, see Configure an API key as an environment variable (to be unpublished and merged into Configure an API key).

Call models using the OpenAI-like method

The LlamaIndex OpenAI-like wrapper lets you call LLMs from Model Studio that support OpenAI compatibility. For a complete list of models, see List of models supported in OpenAI-compatible mode. For information about calling fees, input and output limits, and other details, see Model overview.

Before you start, install the LlamaIndex core component and the OpenAI-like component:

pip install llama-index-core
pip install llama-index-llms-openai-like

Modify global settings (optional)

import os
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike

# Replace the default LLM used by LlamaIndex with a Model Studio model.
Settings.llm = OpenAILike(
    model="qwen-plus",
    api_base="https://dashscope.aliyuncs.com/compatible-mode/v1",
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    is_chat_model=True
)

LLM call example

import os
from llama_index.llms.openai_like import OpenAILike

# The model to call.
llm = OpenAILike(
    model="qwen-plus",
    api_base="https://dashscope.aliyuncs.com/compatible-mode/v1",
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    is_chat_model=True
)

response = llm.complete("Can you recommend a 5-day travel itinerary for the Jiangsu-Zhejiang-Shanghai region?")
print(response)

For a detailed description and the complete API Reference, see the official LlamaIndex OpenAILike API Reference.

Call models using the DashScope method

This method supports all text generation models in Model Studio. For a complete list and calling fees, see Model overview. Deployed models are also supported. For more information, see Deploy models.

Before you start, install the LlamaIndex core component and DashScopeLLM:

pip install llama-index-core
pip install llama-index-llms-dashscope

Modify global settings (optional)

import os
from llama_index.core import Settings
from llama_index.llms.dashscope import DashScope

# Replace the default LLM used by LlamaIndex with a Model Studio model.
Settings.llm = DashScope(model_name="qwen-plus", api_key=os.getenv("DASHSCOPE_API_KEY"))

LLM call example

import os
from llama_index.llms.dashscope import DashScope

llm = DashScope(model_name="qwen-plus", api_key=os.getenv("DASHSCOPE_API_KEY"))
response = llm.complete("Can you recommend a 5-day travel itinerary for the Jiangsu-Zhejiang-Shanghai region?")
print(response)

For a detailed description and more examples, see the official LlamaIndex DashScope LLM Examples. For the complete API Reference, see the official LlamaIndex DashScope API Reference.