Multi-turn Conversations: Correctly Passing Tool Call History
End-to-end practice—from knowledge base creation to multi-turn API calls—demonstrating how to pass tool call history to avoid redundant retrieval
NoteThis guide demonstrates how to pass tool call history across multi-turn conversations, enabling the model to answer continuously based on prior retrieval results and avoid redundant searches. All steps use the same sample document set for reproducible results.
Principle: Why Context Is Lost
RAG Agents are stateless services: each request is independent, and the server does not retain conversation context. A single response typically involves multiple stages:
User Query ──→ Reasoning ──→ Tool Call for Retrieval ──→ Retrieval Results ──→ Sufficient Information?
▲ │
└────────────────────────── No ──────────────────────────────────┤
│ Yes
▼
Final Answer Generation
If only textual history (user query + assistant reply) is passed in the next turn, the model cannot see intermediate tool calls or know what has already been retrieved—leading to redundant retrieval of the same question. This wastes tokens and may yield inconsistent answers.
The correct approach is to include the full tool call chain (assistant.tool_calls + corresponding tool return results) as part of the message history, allowing the model to build upon existing retrieval results.
Prepare Sample Documents
This guide uses the Bailian Technical Documentation Sample, containing 92 official technical documents from the Bailian platform (API usage, model descriptions, best practices, etc.). Download, extract, and upload them to your knowledge base to begin. For demonstration, we’ll use the document “First-Time Calling of Qwen API” to illustrate multi-turn follow-up:
- Turn 1: “How do I call the Qwen API?” (triggers multi-step tool-based retrieval)
- Turn 2: “How do I configure my API Key as an environment variable?” (verifies whether history is reused and redundant retrieval is avoided)
You may also use your own documents—the steps and outcomes remain similar.
End-to-End Workflow
1. Create a Knowledge Base and Upload Sample Documents
- Go to Data Ingestion → Knowledge Management, click Create, and select an edition
- Select Document Search as the knowledge base type and Basic Document Q&A as the use case
- Upload the sample documents, complete chunking and vectorization, and wait until the document status changes to Parsing Completed
See Creating a Knowledge Base and Document Management for details.
2. Create a Knowledge Q&A Service and Bind the Knowledge Base
- Go to Knowledge Services → Knowledge Q&A and click Create
- Enter a service name and proceed to the configuration page after successful creation
- Click + Add, then bind the knowledge base created in Step 1
- Select a generation model and retrieval mode (Multi-turn Intelligent Retrieval is recommended for multi-turn conversations), then click Publish
See Knowledge Q&A for details.
3. Retrieve API Parameters
In the Q&A service list, click Configure for the target service, switch to the Publish Channels tab, and click API Debugging in the Invocation Method section to obtain the following parameters:
| Parameter | Where to Find It | Example |
|---|---|---|
| endpoint | Interface URL on the API Debugging page; {workspace_id} is your workspace ID | https://{workspace_id}.cn-beijing.maas.aliyuncs.com/api/v2/apps/knowledge/chat |
| agent_id | Request parameter on the API Debugging page—the Q&A service ID | aid-xxxxxxxx |
| API Key | Created on the Settings → API Key page | sk-xxxxxxxx |
Noteagent_id uniquely identifies a knowledge Q&A service—each service corresponds to one agent_id.
4. First Turn API Call
Use the retrieved parameters to make your first request. The request structure is as follows:
curl -X POST 'https://{workspace_id}.cn-beijing.maas.aliyuncs.com/api/v2/apps/knowledge/chat' \
-H 'Authorization: Bearer $API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"input": {
"messages": [{"role": "user", "content": "How do I call the Qwen API?"}]
},
"parameters": {
"agent_options": {"agent_id": "aid-xxxxxxxx"}
},
"stream": true
}'
Internally, the Agent executes multi-step tool calls to retrieve from the knowledge base and returns the final answer via streaming. You must extract the tool call history from the SSE stream—see Extracting Tool History from SSE Stream below.
5. Second Turn: Follow-Up with Full Tool History
For the second query—“How do I configure my API Key as an environment variable?”—pass the full history from Turn 1, including the tool call chain:
curl -X POST 'https://{workspace_id}.cn-beijing.maas.aliyuncs.com/api/v2/apps/knowledge/chat' \
-H 'Authorization: Bearer $API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"input": {
"messages": [
{"role": "user", "content": "How do I call the Qwen API?"},
{"role": "assistant", "content": "", "tool_calls": [{"id":"call_956...","function":{"name":"semantic_search",...}}]},
{"role": "tool", "tool_call_id": "call_956...", "content": "Retrieved 5 relevant chunks..."},
{"role": "assistant", "content": "Based on the document \"First-Time Calling of Qwen API\"..."},
{"role": "user", "content": "How do I configure my API Key as an environment variable?"}
]
},
"parameters": {"agent_options": {"agent_id": "aid-xxxxxxxx"}},
"stream": true
}'
WarningThe tool_call_id must match exactly the value returned in the first turn’s streaming response—not generated manually. The model uses it to associate each tool result with its corresponding call.
6. Effect Comparison
Using the “First-Time Calling of Qwen API” document as a test case: After Turn 1 (“How do I call the Qwen API?”), Turn 2 asks “How do I configure my API Key as an environment variable?” Here's how two history-passing strategies compare:
| Full Tool History Passed | Text-Only History Passed | |
|---|---|---|
| Tool Calls in Turn 2 | 0 — direct answer using prior results | Redundant retrieval (2 semantic_search calls observed) |
| Response Speed | Fast — no retrieval overhead | Slower — full retrieval re-executed |
| Answer Consistency | Consistent — based on identical retrieved chunks | Inconsistent — new retrieval may yield different chunks |
Real-world measurement: Turn 1 executed 2 tool calls (semantic_search → obtain_file) generating 6 history messages. When Turn 2 passed full history (7 messages), the model answered directly with 0 tool calls. With text-only history (3 messages), it triggered 2 new semantic_search calls. Exact round counts and message numbers depend on document structure and query matching—but the conclusion holds reliably: passing full history avoids redundant retrieval.
NotePassing full history increases input length (including tool outputs), but eliminates retrieval overhead. Text-only history yields shorter inputs yet incurs full retrieval latency. While trade-offs exist, passing full history delivers superior speed and consistency for multi-turn follow-ups.
Extracting Tool History from SSE Stream
Agent streaming uses Server-Sent Events (SSE). Each data frame is data:{...}, and the relevant path is obj.output.choices[0].message. The field message.extra.step_change marks state transitions—focus only on three event types:
| Event | Identification Condition | Data to Extract |
|---|---|---|
| Tool Call | step_change == "tool_calling" | tool_calls array (all parallel calls in this round) |
| Tool Return | step_change == "tool_return" and role == "tool" | content + tool_call_id |
| Final Answer | Between generation_start and generation_end | Accumulate all content fragments |
State Flow
plan_start (reasoning begins) ──→ plan_end ──→ tool_calling (dispatch tool calls) ──→ tool_return (tools return results) ──→ Need more info?
▲ │
└──────────────────────────────────────── Yes ───────────────────────────────────────────────────────────────────────────┤
│ No
▼
generation_start ──→ generation_end
The Agent may perform multiple tool-call rounds until sufficient information is gathered before generating the final answer.
Message Format and Key Rules
Extracted history messages must be ordered by time and conform to these roles and fields:
| Role | Required Fields | Description |
|---|---|---|
user | role, content | User input |
assistant (tool call) | role, content (empty), tool_calls | All parallel tool calls in this round; function.arguments is a JSON string |
tool | role, content, tool_call_id | Tool result; tool_call_id must match tool_calls[].id |
assistant (final answer) | role, content | Model’s final answer |
Messages must be strictly time-ordered:
user query ──→ assistant (tool_calls) ──→ tool × N ──→ assistant (tool_calls) ──→ tool × N ──→ assistant final answer ──→ user next turn
Multiple tool-call rounds repeat the assistant → tool pattern. Each new turn starts with a fresh user message and repeats the full flow—including all prior history.
Complete Python Implementation
The code below implements the full workflow: extracting history from SSE, auto-passing it across turns, and comparing outcomes. It’s tested against the sample documents and prompts used in this guide.
Configuration
import json
import requests
# Retrieved from the API Debugging page of your Q&A service in the console
API_URL = "https://{workspace_id}.cn-beijing.maas.aliyuncs.com/api/v2/apps/knowledge/chat"
API_KEY = "sk-xxxxxxxx" # Workspace-scoped API Key (starts with "sk-ws-")
AGENT_ID = "aid-xxxxxxxx" # Knowledge Q&A service ID
Extract Tool History from SSE Stream
class ConversationTracker:
"""Tracks a single Agent response, extracting full message sequence from streaming events."""
def __init__(self):
self.messages = [] # Extracted message sequence (tool calls + returns)
self.tool_call_rounds = 0 # Number of tool-call rounds (for effect comparison)
self.answer = "" # Final answer
self._generating = False
def process_sse_line(self, line: str):
if not line.startswith("data:"):
return
obj = json.loads(line[len("data:"):].strip())
choices = obj.get("output", {}).get("choices", [])
if not choices:
return
msg = choices[0].get("message", {})
step_change = msg.get("extra", {}).get("step_change", "")
role = msg.get("role", "")
content = msg.get("content", "") or ""
# Event 1: Tool call dispatch (step_change == "tool_calling") → extract tool_calls
if step_change == "tool_calling" and msg.get("tool_calls"):
self.tool_call_rounds += 1
self.messages.append({
"role": "assistant", "content": "",
"tool_calls": [{"id": tc["id"], "type": "function",
"function": {"name": tc["function"]["name"],
"arguments": tc["function"]["arguments"]}}
for tc in msg["tool_calls"]],
})
# Event 2: Tool return (step_change == "tool_return", role == "tool") → extract content + tool_call_id
elif step_change == "tool_return" and role == "tool":
self.messages.append({"role": "tool",
"tool_call_id": msg.get("tool_call_id", ""),
"content": content})
# Event 3: Final answer (generation_start → content stream → generation_end) → accumulate content
elif step_change == "generation_start":
self._generating = True
self.answer = content
elif self._generating and content:
self.answer += content
elif step_change == "generation_end":
self.answer += content
self._generating = False
def get_response_messages(self):
"""Returns the full extracted message sequence (including final answer)."""
result = list(self.messages)
if self.answer:
result.append({"role": "assistant", "content": self.answer})
return result
Multi-Turn Conversation Manager
class MultiTurnChat:
"""Manages multi-turn history, automatically carrying the full tool-call chain."""
def __init__(self):
self.history = []
def ask(self, question: str) -> str:
"""Sends a question, auto-appends full history (including tool chain), returns final answer."""
self.history.append({"role": "user", "content": question})
sent = len(self.history)
tracker = self._call_agent(self.history)
# Append this turn’s tool chain and final answer to history
self.history.extend(tracker.get_response_messages())
print(f" Sent {sent} messages; {tracker.tool_call_rounds} tool-call rounds")
return tracker.answer
def _call_agent(self, messages):
headers = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
body = {
"input": {"messages": messages},
"parameters": {"agent_options": {"agent_id": AGENT_ID}},
"stream": True,
}
tracker = ConversationTracker()
resp = requests.post(API_URL, headers=headers, json=body, stream=True, timeout=180)
for chunk in resp.iter_lines():
if chunk:
tracker.process_sse_line(chunk.decode("utf-8").strip())
return tracker
Effect Comparison: Full History vs. Text-Only History
chat = MultiTurnChat()
# Turn 1
print("[Turn 1] How do I call the Qwen API?")
chat.ask("How do I call the Qwen API?")
# Turn 2 · Full tool history (auto-carried)
print("\n[Turn 2 · Full tool history]")
chat_full = MultiTurnChat()
chat_full.history = list(chat.history)
chat_full.ask("How do I configure my API Key as an environment variable?")
# Turn 2 · Text-only history (drop tool calls, keep only user + assistant replies)
print("\n[Turn 2 · Text-only history]")
text_only = [m for m in chat.history
if m["role"] == "user" or (m["role"] == "assistant" and not m.get("tool_calls"))]
chat_text = MultiTurnChat()
chat_text.history = text_only
chat_text.ask("How do I configure my API Key as an environment variable?")
Sample output:
[Turn 1] How do I call the Qwen API?
Sent 1 message; 2 tool-call rounds
[Turn 2 · Full tool history]
Sent 7 messages; 0 tool-call rounds ← Reused; no redundant retrieval
[Turn 2 · Text-only history]
Sent 3 messages; 2 tool-call rounds ← Redundant retrieval
Important Notes
Controlling History Length
Tool return content can be lengthy; cumulative history may exceed context limits. Recommended mitigation strategies:
- Truncate older tool returns: Keep full tool content only for the most recent K rounds; truncate earlier
toolmessages to first N characters - Sliding window: For histories >3 turns, replace earlier tool results with summaries
def truncate_history(messages, max_tool_len=2000, keep_recent=2):
"""Truncates tool return content from earlier rounds."""
user_indices = [i for i, m in enumerate(messages) if m["role"] == "user"]
if len(user_indices) <= keep_recent:
return messages
cutoff = user_indices[-keep_recent]
result = []
for i, msg in enumerate(messages):
if i < cutoff and msg["role"] == "tool" and len(msg["content"]) > max_tool_len:
msg = {**msg, "content": msg["content"][:max_tool_len] + "\n...(truncated)"}
result.append(msg)
return result
Content Not Required in History
| Content | Reason |
|---|---|
reasoning_content | Internal reasoning—useful for UI display, but unnecessary for re-injection |
extra field | Streaming protocol metadata—used only for client-side rendering |
usage | Token usage stats—relevant only for monitoring and billing |
tool_call_chunks | Streaming fragments—already merged into tool_calls |
Common Questions
What happens if I omit tool call history? The model loses retrieval context and redundantly re-searches identical queries—degrading answer quality. Real-world test: full history enables zero tool calls on follow-up; omission triggers full re-retrieval.
Can I skip intermediate tool calls and send only the final answer?
It works functionally but is strongly discouraged. Sending only user + assistant(final answer) strips the model of memory about knowledge base content, significantly weakening follow-up performance.
Does order matter for parallel tool returns?
Order doesn’t affect functionality—the model matches results via tool_call_id. However, all tool messages must appear immediately after their corresponding assistant(tool_calls) message.