Using session state in LangGraph
Use AgentRun's OTSCheckpointSaver to automatically persist LangGraph checkpoints to Tablestore. This enables state recovery across requests, allowing conversations to resume from breakpoints without modifying your graph logic. Learn how to integrate OTSCheckpointSaver, manage checkpoints, and deploy your agent to a production environment.
Prerequisites
-
You have created a memory store instance of the Tablestore (OTS) type and enabled the session state feature. If you have not created one, see Create and manage memory stores.
-
You have installed the AgentRun SDK and LangGraph.
pip install agentrun langgraph -
You have set the
MEMORY_COLLECTION_NAMEenvironment variable to the name of your memory store.
The session state feature, once enabled, cannot be disabled. Confirm your choice before proceeding.
Quick start
After meeting the prerequisites, verify that LangGraph checkpoints persist to OTS for multi-turn conversations by using the following example.
Install dependencies:
pip install "agentrun-sdk[server,langgraph]==0.0.31" langgraph langchain-core langchain-openai
The following is the complete example code:
import os
from typing import Annotated, Any, TypedDict
from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
from langchain_openai import ChatOpenAI
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import add_messages
from agentrun.conversation_service import SessionStore
from agentrun.conversation_service.adapters import OTSCheckpointSaver
# Step 1: Initialize SessionStore and create data tables
store = SessionStore.from_memory_collection("your-memory-collection-name")
store.init_langgraph_tables()
# Step 2: Create an OTSCheckpointSaver instance
checkpointer = OTSCheckpointSaver(store)
# Step 3: Define the graph
class ChatState(TypedDict):
messages: Annotated[list[BaseMessage], add_messages]
llm = ChatOpenAI(
model="qwen-max",
api_key=os.environ.get("DASHSCOPE_API_KEY"),
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)
def chat_node(state: ChatState) -> dict[str, Any]:
response = llm.invoke(state["messages"])
return {"messages": [response]}
graph = StateGraph(ChatState)
graph.add_node("chat", chat_node)
graph.add_edge(START, "chat")
graph.add_edge("chat", END)
app = graph.compile(checkpointer=checkpointer)
# Step 4: Run a multi-turn conversation
config = {"configurable": {"thread_id": "my-thread-1"}}
result1 = app.invoke(
{"messages": [HumanMessage(content="Hi, my name is Bob.")]},
config=config,
)
print(f"Message count after the first turn: {len(result1['messages'])}")
result2 = app.invoke(
{"messages": [HumanMessage(content="What is my name?")]},
config=config,
)
print(f"Message count after the second turn: {len(result2['messages'])}")
# Step 5: Verify the output
# Expected output:
# Message count after the first turn: 2
# Message count after the second turn: 4
After running the code, the second turn of the conversation automatically restores the state from the first turn, as indicated by the cumulative message count. Even if the program restarts, the historical state persists as long as you use the same thread_id for subsequent calls.
Integration guide
An agent provides fine-grained, stateful context management for a stateless LLM. Persisting session state is critical for building a production-grade agent, regardless of the framework you use.
LangGraph saves a state snapshot (checkpoint) by calling checkpointer.put() after each step of a graph's execution. OTSCheckpointSaver is an OTS implementation of LangGraph's BaseCheckpointSaver that you pass to StateGraph.compile(checkpointer=...) to persist checkpoints to OTS without modifying any graph logic.
Checkpoint data is stored in three parts in OTS:
|
Data |
OTS table |
Description |
|
Checkpoint metadata |
checkpoint |
Contains the checkpoint ID, parent ID, metadata, and so on. |
|
Channel values |
checkpoint_blobs |
The value of each channel in the graph state, such as the |
|
Pending writes |
checkpoint_writes |
Intermediate write records used for graph execution recovery. |
OTSCheckpointSaver supports the following features:
-
Checkpoint persistence: Automatically persists the complete execution state of the graph (including messages, custom fields, and more) to OTS.
-
State recovery: Subsequent calls with the same
thread_idautomatically restore the last state from OTS, enabling multi-turn conversations across requests. -
Session synchronization: When an
agent_idis set, each checkpoint write automatically syncs a session record to theconversationtable, which supportslist_sessionsqueries. -
Thread deletion: A single call deletes all checkpoint data and associated session records for a thread.
Initialize SessionStore
SessionStore is the core entry point for the Conversation Service. When initialized with a memory store name, the SDK automatically does the following:
-
Calls the AgentRun API to get the memory store configuration.
-
Extracts the Tablestore endpoint and
instance_namefrom the configuration. -
Builds an OTS client.
from agentrun.conversation_service import SessionStore
store = SessionStore.from_memory_collection("your-memory-collection-name")
After initialization, call init_langgraph_tables() to create the required database tables and indexes. This method is idempotent and skips creation if a table or index already exists, so you can safely call it at each startup.
store.init_langgraph_tables()
Asynchronous initialization
All methods of SessionStore have asynchronous versions (the method name is appended with the _async suffix):
store = await SessionStore.from_memory_collection_async(
"your-memory-collection-name")
await store.init_langgraph_tables_async()
Create an OTSCheckpointSaver
OTSCheckpointSaver is the OTS implementation of LangGraph's BaseCheckpointSaver. After creating it, you can pass it to StateGraph.compile().
from agentrun.conversation_service.adapters import OTSCheckpointSaver
checkpointer = OTSCheckpointSaver(store)
The following table describes the parameters.
|
Parameter |
Type |
Required |
Description |
|
session_store |
SessionStore |
Yes |
The SessionStore instance. |
|
agent_id |
str |
No |
The agent ID. If set, checkpoint writes automatically synchronize records in the |
|
user_id |
str |
No |
The default user ID. You can override this value for each call by using |
Specify agent_id and user_id:
checkpointer = OTSCheckpointSaver(
store,
agent_id="my_agent",
user_id="default_user",
)
# Subsequent put operations will automatically synchronize conversation records.
# You can query them by using list_sessions.
sessions = store.list_sessions("my_agent", "default_user")
Session synchronization
When agent_id is set, OTSCheckpointSaver automatically creates or updates a session record (session_id = thread_id, framework = "langgraph") in the conversation table each time put is called. External services can query LangGraph sessions through list_sessions or search_sessions.
User_id priority
user_id is resolved in the following order of priority:
-
config["metadata"]["user_id"](Dynamically passed in for each call and has the highest priority) -
Constructor parameter
user_id -
Default value
"default"
In server scenarios, user_id is typically obtained from the HTTP header and passed through config metadata.
Build the graph and use the checkpointer
Pass OTSCheckpointSaver to the checkpointer parameter of StateGraph.compile():
config = {
"configurable": {"thread_id": thread_id},
"metadata": {"user_id": user_id},
}
await app.ainvoke({"messages": [...]}, config=config)
After compilation, thread_id is used to distinguish between different conversation threads. Calls with the same thread_id automatically restore the previous state:
config = {"configurable": {"thread_id": "my-thread-1"}}
# First turn: new thread
result1 = await app.ainvoke(
{"messages": [HumanMessage(content="Hello")]},
config=config,
)
# Second turn: automatically restores the state from the first turn
result2 = await app.ainvoke(
{"messages": [HumanMessage(content="Let's continue")]},
config=config,
)
# result2["messages"] contains all messages from both the first and second turns.
Checkpoint management
Query a checkpoint
You can use get_tuple / aget_tuple to retrieve the latest checkpoint of a specified thread:
config = {"configurable": {"thread_id": "my-thread-1"}}
checkpoint_tuple = await checkpointer.aget_tuple(config)
if checkpoint_tuple:
cp = checkpoint_tuple.checkpoint
print(f"checkpoint_id: {cp['id']}")
messages = cp["channel_values"].get("messages", [])
print(f"Message count: {len(messages)}")
print(f"metadata: {checkpoint_tuple.metadata}")
You can also specify checkpoint_id to query a specific version:
config = {
"configurable": {
"thread_id": "my-thread-1",
"checkpoint_id": "specific-checkpoint-id",
}
}
checkpoint_tuple = await checkpointer.aget_tuple(config)
List checkpoint history
Use list / alist to list the checkpoint history of a thread (in descending order of checkpoint_id):
config = {"configurable": {"thread_id": "my-thread-1"}}
async for cp_tuple in checkpointer.alist(config, limit=10):
print(f" checkpoint_id={cp_tuple.checkpoint['id']}")
Supports the filter parameter to filter by metadata, and the before parameter for pagination:
async for cp_tuple in checkpointer.alist(
config,
filter={"source": "loop"},
limit=5,
):
print(cp_tuple.metadata)
Delete a thread
Deleting a thread removes all associated checkpoint data. If agent_id is set, it also cascade-deletes the conversation records in the conversation table:
await checkpointer.adelete_thread("my-thread-1")
Roll back to a historical checkpoint
Restore to a previous state by specifying the checkpoint_id:
# List historical checkpoints
config = {"configurable": {"thread_id": "my-thread-1"}}
async for cp_tuple in checkpointer.alist(config, limit=5):
print(f"checkpoint_id={cp_tuple.checkpoint['id']}")
# Restore from a specific checkpoint
config_with_cp = {
"configurable": {
"thread_id": "my-thread-1",
"checkpoint_id": "target-checkpoint-id",
}
}
result = await app.ainvoke(
{"messages": [HumanMessage(content="Continue")]},
config=config_with_cp,
)
Deploy with AgentRunServer
In production, a LangGraph Agent is typically deployed as an HTTP service. Save the following server code as conversation_service_langgraph_server.py:
import os
import uuid
from typing import Annotated, Any, TypedDict
from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
from langchain_openai import ChatOpenAI
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import add_messages
from agentrun import AgentRequest
from agentrun.conversation_service import SessionStore
from agentrun.conversation_service.adapters import OTSCheckpointSaver
from agentrun.server import AgentRunServer
AGENT_ID = "langgraph_chat_server"
class ChatState(TypedDict):
messages: Annotated[list[BaseMessage], add_messages]
llm = ChatOpenAI(
model="qwen-max",
api_key=os.environ["DASHSCOPE_API_KEY"],
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)
def chat_node(state: ChatState) -> dict[str, Any]:
response = llm.invoke(state["messages"])
return {"messages": [response]}
# Initialization
store = SessionStore.from_memory_collection(
os.environ["MEMORY_COLLECTION_NAME"])
store.init_langgraph_tables()
checkpointer = OTSCheckpointSaver(store, agent_id=AGENT_ID)
graph = StateGraph(ChatState)
graph.add_node("chat", chat_node)
graph.add_edge(START, "chat")
graph.add_edge("chat", END)
app = graph.compile(checkpointer=checkpointer)
# Core handler function
async def invoke_agent(req: AgentRequest):
# Get thread_id and user_id from HTTP headers
headers = dict(req.raw_request.headers) if req.raw_request else {}
thread_id = (
headers.get("x-agentrun-session-id")
or f"thread_{uuid.uuid4().hex[:8]}"
)
user_id = headers.get("x-agentrun-user-id") or "default_user"
config = {
"configurable": {"thread_id": thread_id},
"metadata": {"user_id": user_id},
}
# Extract the last user message
last_user_text = ""
for msg in reversed(req.messages):
if msg.role == "user":
last_user_text = msg.content or ""
break
if not last_user_text:
yield "Please enter a question."
return
# Call LangGraph for streaming output
async for event in app.astream(
{"messages": [HumanMessage(content=last_user_text)]},
config=config,
stream_mode="values",
):
messages = event.get("messages", [])
if messages:
last_msg = messages[-1]
if isinstance(last_msg, AIMessage) and last_msg.content:
yield last_msg.content
# Start the server
if __name__ == "__main__":
server = AgentRunServer(
invoke_agent=invoke_agent,
memory_collection_name=os.environ["MEMORY_COLLECTION_NAME"],
)
server.start(port=9000)
The following are the header conventions for client requests:
|
Header |
Description |
Required |
|
X-AgentRun-Session-ID |
Corresponds to LangGraph's |
No (If not provided, the system automatically generates a new thread). |
|
X-AgentRun-User-ID |
The user ID, which is written to |
No (Defaults to |
To test locally, ensure the port matches the port configuration in s.yaml:
curl -X POST http://localhost:9000/openai/v1/chat/completions \
-H "Content-Type: application/json" \
-H "X-AgentRun-Session-ID: my-thread-1" \
-H "X-AgentRun-User-ID: user-1" \
-d '{"model":"qwen-max","stream":true,"messages":[{"role":"user","content":"Hello"}]}'
Deploy to the AgentRun cloud platform
Install deployment tools
Install Serverless Devs (the s command) via npm:
npm install -g @serverless-devs/s
After installation, verify it:
s --version
Configure your Alibaba Cloud credentials. Run the following command and enter your AccessKey ID and AccessKey Secret as prompted:
s config add
Remember the name you set for the key pair during configuration, such as default. This name is used in the access field of s.yaml.
The s build command requires Docker to build Linux x64-compatible dependency packages. Install Docker Desktop or a compatible environment (such as Podman or Lima) beforehand.
Prepare configuration files
Ensure that the root directory of your project contains a requirements.txt file:
agentrun-sdk[server,langgraph]==0.0.31
python-dotenv>=1.0.0
Create a .fcignore file to exclude sensitive files and the local environment:
.env
.venv
Create the s.yaml configuration file in the project root directory:
edition: 3.0.0
name: my-conversation-service
access: 'default' # Replace with the name of the key pair you configured with 's config add'.
vars:
region: 'cn-hangzhou'
agent_name: 'my-langgraph-agent' # Agent name
role: 'acs:ram::${config("AccountID")}:role/AliyunAgentRunDefaultRole'
memory_collection_name: 'your-memory-collection-name' # Replace with your memory collection name.
api_key: 'your-dashscope-api-key' # Replace with your DashScope API key.
resources:
agent:
component: agentrun
props:
region: ${vars.region}
agent:
name: ${vars.agent_name}
description: 'LangGraph Conversation Service'
role: ${vars.role}
code:
src: ./
language: python3.12
command:
- python3
- conversation_service_langgraph_server.py
cpu: 1.0
memory: 2048
port: 9000
instanceConcurrency: 200
internetAccess: true
environmentVariables:
PYTHONPATH: /code/python:/opt/python:/code:/code/python
MEMORY_COLLECTION_NAME: '${vars.memory_collection_name}'
DASHSCOPE_API_KEY: '${vars.api_key}'
logConfig: auto
endpoints:
- name: prod
version: LATEST
description: 'Production endpoint'
Key configuration fields:
|
Field |
Description |
|
access |
The name of the key pair you set with |
|
vars.role |
The RAM role for the agent at runtime. Go to the RAM role management page to view or create it. |
|
code.command |
Startup command: Modify the server filename based on the framework you use. The example in this article is |
|
environmentVariables |
Environment variables for the cloud runtime that replace the local |
Build and deploy
The cloud environment is Linux x64. Use the s build command to install compatible dependencies in Docker:
s build --use-docker
This command pulls the Function Compute runtime image and runs pip install within it to install dependencies into the python/ directory.
After the build completes, deploy the agent:
s deploy
After a successful deployment, the output includes the agent's access URL:
endpoints:
-
name: prod
url: https://<agent-id>.agentrun-data.cn-hangzhou.aliyuncs.com/agent-runtimes/<runtime-id>/endpoints/prod/invocations
status: READY
Test the deployment
Send a request using the URL from the deployment output. Replace <your-endpoint-url> with the actual URL from the s deploy output:
curl -X POST https://<your-endpoint-url>/openai/v1/chat/completions \
-H "Content-Type: application/json" \
-H "X-AgentRun-Session-ID: my-thread-1" \
-H "X-AgentRun-User-ID: user-1" \
-d '{"model":"qwen-max","stream":true,"messages":[{"role":"user","content":"Hello"}]}'
FAQ
Idempotent initialization
Yes. init_langgraph_tables() is idempotent. It skips creation if the tables or indexes already exist. We recommend calling it at application startup to ensure the required resources are ready.
Relationship between thread_id and session_id
In LangGraph, thread_id is used to distinguish between different conversation threads. When session synchronization is enabled (by setting agent_id), OTSCheckpointSaver maps thread_id to session_id in the conversation table. The two have the same value.
When to set agent_id
It depends on your use case:
-
If not set: Checkpoint data still persists and restores normally, but you cannot query the session list by using
list_sessionsorsearch_sessions. This approach is suitable for simple scenarios or for applications that do not require session management. -
If set (recommended): Each checkpoint write automatically synchronizes a record in the conversation table. This approach is suitable for production applications that need to display or search session lists.
LangGraph vs. LangChain
|
Dimension |
LangGraph (OTSCheckpointSaver) |
LangChain (OTSChatMessageHistory) |
|
Persistence granularity |
Complete graph state snapshot (includes all channel values). |
Single messages (one event per message). |
|
State recovery |
Automatically restores all state fields, including messages and custom fields. |
Restores only the message list. |
|
Use cases |
Complex agent graphs with multiple nodes, conditional branches, and tool calls. |
Simple conversational chains (Prompt + LLM). |
|
Initialization method |
|
|
Supported models
The Conversation Service does not restrict your choice of model. The example uses Qwen via the DashScope API, but you can replace it with any LangChain-supported chat model, such as OpenAI GPT, Anthropic Claude, or Google Gemini. The LLM configuration in LangGraph nodes determines the model choice and is independent of the Conversation Service.
Troubleshooting and common issues
You may encounter the following common issues during integration:
|
Error scenario |
Error message |
Solution |
|
Authentication failure |
|
Run |
|
Network timeout |
|
In the AgentRun console, check the endpoint configuration in the memory store and confirm network reachability. If you run in a VPC, use the internal endpoint. You can also add retry logic at the application layer. |
|
Missing environment variable |
|
Run |
|
Memory store not found |
|
Go to the AgentRun console to verify the memory store name, which is case-sensitive. Ensure that it matches the value of the |
|
Checkpoint recovery failure |
The multi-turn conversation fails to restore the historical state. |
Verify that multiple calls use the same |
We recommend adding exception handling for critical operations in a production environment:
from langgraph.graph import StateGraph
try:
result = await app.ainvoke(
{"messages": [HumanMessage(content="Hello")]},
config=config,
)
except Exception as e:
print(f"Graph execution failed: {e}")
# Check the checkpoint status
cp = await checkpointer.aget_tuple(config)
if cp:
print(f"Last successful checkpoint: {cp.checkpoint['id']}")
raise