Optimization

Continuously improve application quality through version comparison, debugging previews, user feedback, and high-quality evaluation data

Optimization supports multi-version comparison, prompt optimization based on debugging results and feedback, and prompt optimization based on high-quality evaluation data. This topic describes how to generate, review, and adopt optimized prompts based on debugging results and human feedback.

Before You Begin

Before you start, make sure the application has been created and published successfully. Also prepare test questions that can expose problems with the application—for example, incomplete answers, an unexpected tone, failure to follow formatting requirements, or compliance risks.

Open Optimization

Go to the Optimization page and click Select the Application to Optimize to start an optimization task.

Multi-Version Comparison

Review and Refine the Current Prompt

In the Prompt Optimization area on the left, review the application's current prompt and confirm that the basics are complete—role definition, task objectives, response rules, boundary conditions, and output format. You can edit the prompt directly in the editor if changes are needed.

Start a Version Comparison

Click Comparison Mode to open the version-comparison debugging dialog. By default it shows one baseline group and one control group, and you can add up to two control groups. The initial configuration of the baseline and control groups matches the current draft configuration of the application. Click Settings in the top-right corner to view the baseline prompt or adjust the control-group prompts.

Debug Across Versions

After making adjustments, enter a question and start a debugging task. The baseline and control groups each produce a response, so you can compare how different prompt versions behave. You can run multiple rounds of debugging, and modify the configuration at any time before continuing.

Adopt a Configuration

Once you have identified the best-performing version, click Adopt This Group's Configuration on that version. The system writes the group's prompt into the prompt editor.

Debugging and Feedback Optimization

Start a Debugging Conversation

Enter test questions in the Debugging Window on the right and send them to view the application's responses. Prioritize scenarios where problems commonly appear in real business use, for example:

• The response omits rules, conditions, or exceptions.
• The tone is stiff and does not match the application's persona.
• The output does not follow the required format, or is verbose and poorly structured.
• The response ignores knowledge base content and is not grounded.
• Safety, compliance, or business-boundary constraints are missing.

You can run multiple rounds of conversation to validate the same prompt from different angles.

Select the Debugging Results to Optimize

Select one or more debugging results depending on the scope of the problem:

• One result: Best for a clearly defined problem that a single case can illustrate. The optimization dialog shows "Current debugging result".
• Multiple results: Best for problems that recur or require an overall judgment. After entering batch-selection mode, select representative results; the optimization dialog shows "Selected debugging results (N)".

Provide Human Feedback

Click Optimize Prompt Based on Debugging Results to open the Optimize Prompt Based on Debugging Results dialog.

In Human Evaluation Feedback, describe the problems with the application's responses and the improvements you expect. Feedback should be specific—state "what is wrong" and "how it should be adjusted". Human feedback supplements business requirements that cannot be inferred directly from the debugging conversation. If you selected multiple debugging results, you can summarize their common problems and shared optimization goal in a single piece of feedback.

Generate and Review the Optimization Result

Click Start Optimization. The system analyzes the problem and generates a new prompt based on:

• The application's current prompt.
• The current or selected debugging conversations.
• The human evaluation feedback.

When generation completes, review the optimization result in the dialog—the comparison between the current prompt and the intelligently optimized result—and decide whether to adopt it.

Adopt the Optimization Result

After confirming that the optimized prompt meets expectations, click Adopt. The system writes the result into the prompt editor.

Validate and Publish

After adopting an optimization result, run debugging again and cover the following scenarios:

• The original problem scenario: Confirm the problem has been resolved.
• Normal scenarios: Confirm existing capabilities have not regressed.
• Boundary scenarios: Confirm abnormal input, insufficient information, or conflicting instructions are handled reasonably.

Once the results meet expectations, click Publish to complete the application version update.

Optimization Tips

• Focus on one main problem at a time: Optimize content completeness, tone, formatting, and compliance in separate batches—it is easier to judge the effect of each change.
• Use real, representative conversations: Turn high-frequency production issues and failed evaluation samples into debugging questions, or use test questions that reliably reproduce the problem.
• Make feedback explicit and constrained: Rather than "answer better", state exactly what must be included, what must not appear, and the expected output form.
• Adopt first, then validate, then publish: Do not judge effectiveness from the optimized text alone—validate actual response quality through debugging.
• Keep humans in the loop: Intelligent optimization may not understand implicit business rules. Review carefully whenever knowledge standards, permissions, safety, or compliance requirements are involved.

FAQ

Does the application take effect immediately after I adopt an optimization result?

No. Adopting only writes the optimized content into the current prompt. You still need to validate it through debugging and click Publish before the new version takes effect.

Should I select one debugging result or several?

Select one result for a single, clearly defined problem. If the same type of problem recurs across multiple questions, select several results to help the system identify the common pattern. Multiple results should focus on the same optimization goal.

How do I write high-quality human feedback?

Use the pattern "symptom + specific gap or error + expected behavior". For example: "The response only covers the general refund policy and omits the exception for home appliances; please list the general rules, applicable conditions, and special-category exceptions in full."

Do I need to debug again after optimizing?

Yes. Prompt changes can affect other response scenarios. Validate the original problem, normal flows, and boundary inputs together, and confirm no new problems were introduced before publishing.