DocsICP FilingConsole
Documentation

Build a private knowledge Q&A application with zero code

Updated at:
Copy as MD
Product
My Favorites

Large language models (LLMs) cannot directly answer questions about private knowledge. Model Studio lets you connect your own documents to an agent application—no code required—so users get accurate, grounded answers instead of generic or fabricated responses.

Why LLMs cannot answer questions about private knowledge

The knowledge of large language models (LLMs) comes from their training data, which does not include private, domain-specific, or up-to-date information. Therefore, asking them directly about a specific product or internal knowledge results in inaccurate or fabricated answers.

Retrieval-Augmented Generation (RAG) combines an external knowledge base with a large language model. By retrieving relevant information and providing it to the model, it enables the model to answer based on the given private knowledge. The core principles are as follows:

  1. Data processing and vectorization: Uploaded documents are automatically processed. The system splits them into chunks, converts each chunk into a mathematical vector using an embedding model, and stores the vectors in a vector database.
  2. Retrieval: When a question is received, the system also vectorizes the question, and then retrieves the Top-k document chunks whose vectors are most similar to the question vector from the vector database.
  3. Generation: The system combines the original question with the retrieved document fragments into a prompt, and submits it to the model to generate an answer.

Create a private knowledge Q&A application

This tutorial uses the new agent application (Agent 2.0). In the current console, the application created by default after you click Create Agent is an Agent 2.0 application. To create a legacy (Agent 1.0) application, click Legacy Agent creation entry at the bottom of the creation dialog, and the page will redirect to the legacy Model Studio console.

Create an agent

  1. Create the application: Visit the Application Management page, click Create Agent, enter the Application Name (this tutorial uses "Alibaba Cloud Model Studio Phone Shopping Guide" as an example), and click Create Now. The application configuration page opens automatically after creation.

  2. Select a model: In the Planning area of the application configuration page, click the model selector to select a large language model:

    • Smart mode: The platform automatically balances cost and effectiveness, so you do not need to select a model manually. Recommended for beginners. The AUTO tier automatically routes to an appropriate performance tier based on task complexity.
    • Specify a model: Select a specific model directly. To ensure effective multi-step planning, a model with strong tool-calling capabilities is recommended, such as the 千问-Max series models. Click More Models to select other models.

    NoteUsing large models incurs fees. Alibaba Cloud Model Studio provides a limited-time free quota. You can go to Model Square and click View Details on each model card to check it.

  3. Design the prompt: In the prompt input box in the Planning area, define the role and task of the application. The recommended setting is:

    You are an Alibaba Cloud Model Studio phone shopping guide. Your task is to help customers compare phone specifications, analyze their needs, and provide personalized recommendations.

    Variables can be embedded in the system prompt: click Create Variable in the upper-right corner of the prompt to configure one, and type / to reference a configured variable.

  4. Test the application: Enter a question (such as "What models of Alibaba Cloud Model Studio phones are available?") in the dialog box on the right to ask a question.

    NoteBecause the application lacks knowledge about "Alibaba Cloud Model Studio phones", its answers are rather generic and may even be fabricated. Next, associate a knowledge base with the application.

Create a knowledge base

  1. In the left-side navigation pane, choose Data Ingestion > Knowledge Management to go to the Knowledge Management page, and click Create.

  2. Enter the Name and Description, set the knowledge base type to Document Search, set the use case to Basic Document Q&A, and click Next.

  3. Set the data source to Upload Files, and drag and drop or click to upload a document about "Alibaba Cloud Model Studio phones" (PDF, Word, Markdown, and other formats are supported). You can download the sample file Alibaba Cloud Model Studio Phone Series Product Introduction.docx.

  4. Keep the default index settings and click Create Knowledge Base.

    NoteThe system automatically completes parsing, chunking, vectorization, and index building. The knowledge base can be used for retrieval after its status changes to Ready. For complete configuration details such as knowledge base type, use case, and chunking strategy, see Create and use a knowledge base.

Create and publish a knowledge retrieval service

The new agent uses knowledge bases by associating a knowledge retrieval service. A knowledge retrieval service can contain one or more knowledge bases. Complete creation and publication before associating.

  1. In the left-side navigation pane, choose Knowledge Service > Knowledge Retrieval to go to the Knowledge Retrieval page, click Create Retrieval Service in the upper-right corner, enter a Service Name (this tutorial uses "Alibaba Cloud Model Studio Phone Retrieval"), and click Confirm.

  2. The configuration page opens automatically after creation. Click + Add to bind the "Alibaba Cloud Model Studio phones" knowledge base created in the previous step. When you bind multiple knowledge bases, you can set a weight for each knowledge base. The higher the weight, the higher the priority of its retrieval results in the final ranking.

  3. Enter a query (such as "Which phone has the best camera?") in the chat window on the right. After you confirm that the returned chunks meet expectations, click Publish in the upper-right corner. The retrieval service can be associated with an application only after it is published.

    NoteFor the retrieval service configuration (knowledge base routing, hybrid ranking model, retrieval count, etc.), the defaults are sufficient for this tutorial. For complete details, see Knowledge retrieval.

Associate the knowledge retrieval service with the agent

  1. Go to the Application Management page, locate the created "Alibaba Cloud Model Studio Phone Shopping Guide" application, and click Configure to open the application configuration page.
  2. Find the Retrieval Strategy area under Knowledge Base. On first configuration, the page indicates that no knowledge retrieval service is configured. Click Configure Now to open the service selection panel. You can also click Configure in the upper-right corner of the area to enter.
  3. In the Configure Retrieval Service panel, use the search box to find the published "Alibaba Cloud Model Studio Phone" retrieval service, verify the target based on the service name, version, included knowledge bases, and number of knowledge bases, and then click Link on the right of the target service to associate the knowledge retrieval service with the current application.

Test and publish

  1. Return to the application configuration page and ask again in the dialog box on the right, for example: "Help me pick an Alibaba Cloud Model Studio phone with the best camera performance for under CNY 3,000." With retrieval augmentation, the application can accurately answer shopping questions based on the content in the knowledge base.

  2. Click Publish in the upper-right corner, confirm the version information in the Publish Version dialog box, and click Confirm Publish.

    NoteVersion IDs increment starting from V1.0, and each application can store up to 10 versions. Publishing is a prerequisite for calling the application through the API or SDK.

What's next

  • Optimize RAG performance: Learn how to optimize performance in the query, retrieval, recall, and reranking stages. See Knowledge base optimization.
  • Configure more tools: Configure external tools such as plugins and MCP for the agent to further extend the capabilities of large models. See New agent application (Agent 2.0).
  • Explore more powerful applications: Use workflow applications to orchestrate complex processes, handling multi-step business scenarios with control logic such as conditional branches, loops, and parallel execution.
  • External calls and integration: Integrate the application into your business systems through the DashScope SDK or HTTP requests. See Call an agent application and Invoke a workflow application.

FAQ

Why does the application answer inaccurately, or not answer using the knowledge in the knowledge base?

This can be caused by multiple stages in the RAG pipeline:

  1. Document chunking broke the semantic integrity.
  2. The embedding model failed to accurately capture the core semantics of the text.
  3. The retrieval stage failed to recall the most relevant document fragments.

Try the following tuning options:

  1. Use well-structured documents, such as Markdown or JSON.
  2. When configuring the knowledge base, select the Custom configuration mode, and debug the answer quality by selecting different embedding models and setting different ranking configurations.

How can I make the model explicitly say it does not know when it cannot find an answer in the knowledge base, instead of fabricating one?

You can enable the Agent reply intervention feature and set the intervention reply for when the knowledge base returns no results.