Using session state in LangGraph

Updated at:
Copy as MD

Use AgentRun's OTSCheckpointSaver to automatically persist LangGraph checkpoints to Tablestore. This enables state recovery across requests, allowing conversations to resume from breakpoints without modifying your graph logic. Learn how to integrate OTSCheckpointSaver, manage checkpoints, and deploy your agent to a production environment.

Prerequisites

  • You have created a memory store instance of the Tablestore (OTS) type and enabled the session state feature. If you have not created one, see Create and manage memory stores.

  • You have installed the AgentRun SDK and LangGraph.

    pip install agentrun langgraph
  • You have set the MEMORY_COLLECTION_NAME environment variable to the name of your memory store.

Important

The session state feature, once enabled, cannot be disabled. Confirm your choice before proceeding.

Quick start

After meeting the prerequisites, verify that LangGraph checkpoints persist to OTS for multi-turn conversations by using the following example.

Install dependencies:

pip install "agentrun-sdk[server,langgraph]==0.0.31" langgraph langchain-core langchain-openai

The following is the complete example code:

import os
from typing import Annotated, Any, TypedDict

from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
from langchain_openai import ChatOpenAI
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import add_messages

from agentrun.conversation_service import SessionStore
from agentrun.conversation_service.adapters import OTSCheckpointSaver

# Step 1: Initialize SessionStore and create data tables
store = SessionStore.from_memory_collection("your-memory-collection-name")
store.init_langgraph_tables()

# Step 2: Create an OTSCheckpointSaver instance
checkpointer = OTSCheckpointSaver(store)

# Step 3: Define the graph
class ChatState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]

llm = ChatOpenAI(
    model="qwen-max",
    api_key=os.environ.get("DASHSCOPE_API_KEY"),
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)

def chat_node(state: ChatState) -> dict[str, Any]:
    response = llm.invoke(state["messages"])
    return {"messages": [response]}

graph = StateGraph(ChatState)
graph.add_node("chat", chat_node)
graph.add_edge(START, "chat")
graph.add_edge("chat", END)
app = graph.compile(checkpointer=checkpointer)

# Step 4: Run a multi-turn conversation
config = {"configurable": {"thread_id": "my-thread-1"}}

result1 = app.invoke(
    {"messages": [HumanMessage(content="Hi, my name is Bob.")]},
    config=config,
)
print(f"Message count after the first turn: {len(result1['messages'])}")

result2 = app.invoke(
    {"messages": [HumanMessage(content="What is my name?")]},
    config=config,
)
print(f"Message count after the second turn: {len(result2['messages'])}")

# Step 5: Verify the output
# Expected output:
#   Message count after the first turn: 2
#   Message count after the second turn: 4

After running the code, the second turn of the conversation automatically restores the state from the first turn, as indicated by the cumulative message count. Even if the program restarts, the historical state persists as long as you use the same thread_id for subsequent calls.

Integration guide

An agent provides fine-grained, stateful context management for a stateless LLM. Persisting session state is critical for building a production-grade agent, regardless of the framework you use.

LangGraph saves a state snapshot (checkpoint) by calling checkpointer.put() after each step of a graph's execution. OTSCheckpointSaver is an OTS implementation of LangGraph's BaseCheckpointSaver that you pass to StateGraph.compile(checkpointer=...) to persist checkpoints to OTS without modifying any graph logic.

Checkpoint data is stored in three parts in OTS:

Data

OTS table

Description

Checkpoint metadata

checkpoint

Contains the checkpoint ID, parent ID, metadata, and so on.

Channel values

checkpoint_blobs

The value of each channel in the graph state, such as the messages list, stored as base64-encoded data.

Pending writes

checkpoint_writes

Intermediate write records used for graph execution recovery.

OTSCheckpointSaver supports the following features:

  • Checkpoint persistence: Automatically persists the complete execution state of the graph (including messages, custom fields, and more) to OTS.

  • State recovery: Subsequent calls with the same thread_id automatically restore the last state from OTS, enabling multi-turn conversations across requests.

  • Session synchronization: When an agent_id is set, each checkpoint write automatically syncs a session record to the conversation table, which supports list_sessions queries.

  • Thread deletion: A single call deletes all checkpoint data and associated session records for a thread.

Initialize SessionStore

SessionStore is the core entry point for the Conversation Service. When initialized with a memory store name, the SDK automatically does the following:

  1. Calls the AgentRun API to get the memory store configuration.

  2. Extracts the Tablestore endpoint and instance_name from the configuration.

  3. Builds an OTS client.

from agentrun.conversation_service import SessionStore
store = SessionStore.from_memory_collection("your-memory-collection-name")

After initialization, call init_langgraph_tables() to create the required database tables and indexes. This method is idempotent and skips creation if a table or index already exists, so you can safely call it at each startup.

store.init_langgraph_tables()

Asynchronous initialization

All methods of SessionStore have asynchronous versions (the method name is appended with the _async suffix):

store = await SessionStore.from_memory_collection_async(
    "your-memory-collection-name")
await store.init_langgraph_tables_async()

Create an OTSCheckpointSaver

OTSCheckpointSaver is the OTS implementation of LangGraph's BaseCheckpointSaver. After creating it, you can pass it to StateGraph.compile().

from agentrun.conversation_service.adapters import OTSCheckpointSaver
checkpointer = OTSCheckpointSaver(store)

The following table describes the parameters.

Parameter

Type

Required

Description

session_store

SessionStore

Yes

The SessionStore instance.

agent_id

str

No

The agent ID. If set, checkpoint writes automatically synchronize records in the conversation table.

user_id

str

No

The default user ID. You can override this value for each call by using config["metadata"]["user_id"].

Specify agent_id and user_id:

checkpointer = OTSCheckpointSaver(
    store,
    agent_id="my_agent",
    user_id="default_user",
)
# Subsequent put operations will automatically synchronize conversation records.
# You can query them by using list_sessions.
sessions = store.list_sessions("my_agent", "default_user")

Session synchronization

When agent_id is set, OTSCheckpointSaver automatically creates or updates a session record (session_id = thread_id, framework = "langgraph") in the conversation table each time put is called. External services can query LangGraph sessions through list_sessions or search_sessions.

User_id priority

user_id is resolved in the following order of priority:

  1. config["metadata"]["user_id"] (Dynamically passed in for each call and has the highest priority)

  2. Constructor parameter user_id

  3. Default value "default"

In server scenarios, user_id is typically obtained from the HTTP header and passed through config metadata.

Build the graph and use the checkpointer

Pass OTSCheckpointSaver to the checkpointer parameter of StateGraph.compile():

config = {
    "configurable": {"thread_id": thread_id},
    "metadata": {"user_id": user_id},
}
await app.ainvoke({"messages": [...]}, config=config)

After compilation, thread_id is used to distinguish between different conversation threads. Calls with the same thread_id automatically restore the previous state:

config = {"configurable": {"thread_id": "my-thread-1"}}

# First turn: new thread
result1 = await app.ainvoke(
    {"messages": [HumanMessage(content="Hello")]},
    config=config,
)

# Second turn: automatically restores the state from the first turn
result2 = await app.ainvoke(
    {"messages": [HumanMessage(content="Let's continue")]},
    config=config,
)
# result2["messages"] contains all messages from both the first and second turns.

Checkpoint management

Query a checkpoint

You can use get_tuple / aget_tuple to retrieve the latest checkpoint of a specified thread:

config = {"configurable": {"thread_id": "my-thread-1"}}
checkpoint_tuple = await checkpointer.aget_tuple(config)
if checkpoint_tuple:
    cp = checkpoint_tuple.checkpoint
    print(f"checkpoint_id: {cp['id']}")
    messages = cp["channel_values"].get("messages", [])
    print(f"Message count: {len(messages)}")
    print(f"metadata: {checkpoint_tuple.metadata}")

You can also specify checkpoint_id to query a specific version:

config = {
    "configurable": {
        "thread_id": "my-thread-1",
        "checkpoint_id": "specific-checkpoint-id",
    }
}
checkpoint_tuple = await checkpointer.aget_tuple(config)

List checkpoint history

Use list / alist to list the checkpoint history of a thread (in descending order of checkpoint_id):

config = {"configurable": {"thread_id": "my-thread-1"}}
async for cp_tuple in checkpointer.alist(config, limit=10):
    print(f"  checkpoint_id={cp_tuple.checkpoint['id']}")

Supports the filter parameter to filter by metadata, and the before parameter for pagination:

async for cp_tuple in checkpointer.alist(
    config,
    filter={"source": "loop"},
    limit=5,
):
    print(cp_tuple.metadata)

Delete a thread

Deleting a thread removes all associated checkpoint data. If agent_id is set, it also cascade-deletes the conversation records in the conversation table:

await checkpointer.adelete_thread("my-thread-1")

Roll back to a historical checkpoint

Restore to a previous state by specifying the checkpoint_id:

# List historical checkpoints
config = {"configurable": {"thread_id": "my-thread-1"}}
async for cp_tuple in checkpointer.alist(config, limit=5):
    print(f"checkpoint_id={cp_tuple.checkpoint['id']}")

# Restore from a specific checkpoint
config_with_cp = {
    "configurable": {
        "thread_id": "my-thread-1",
        "checkpoint_id": "target-checkpoint-id",
    }
}
result = await app.ainvoke(
    {"messages": [HumanMessage(content="Continue")]},
    config=config_with_cp,
)

Deploy with AgentRunServer

In production, a LangGraph Agent is typically deployed as an HTTP service. Save the following server code as conversation_service_langgraph_server.py:

import os
import uuid
from typing import Annotated, Any, TypedDict

from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
from langchain_openai import ChatOpenAI
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import add_messages

from agentrun import AgentRequest
from agentrun.conversation_service import SessionStore
from agentrun.conversation_service.adapters import OTSCheckpointSaver
from agentrun.server import AgentRunServer

AGENT_ID = "langgraph_chat_server"

class ChatState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]

llm = ChatOpenAI(
    model="qwen-max",
    api_key=os.environ["DASHSCOPE_API_KEY"],
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
)

def chat_node(state: ChatState) -> dict[str, Any]:
    response = llm.invoke(state["messages"])
    return {"messages": [response]}

# Initialization
store = SessionStore.from_memory_collection(
    os.environ["MEMORY_COLLECTION_NAME"])
store.init_langgraph_tables()
checkpointer = OTSCheckpointSaver(store, agent_id=AGENT_ID)

graph = StateGraph(ChatState)
graph.add_node("chat", chat_node)
graph.add_edge(START, "chat")
graph.add_edge("chat", END)
app = graph.compile(checkpointer=checkpointer)

# Core handler function
async def invoke_agent(req: AgentRequest):
    # Get thread_id and user_id from HTTP headers
    headers = dict(req.raw_request.headers) if req.raw_request else {}
    thread_id = (
        headers.get("x-agentrun-session-id")
        or f"thread_{uuid.uuid4().hex[:8]}"
    )
    user_id = headers.get("x-agentrun-user-id") or "default_user"
    config = {
        "configurable": {"thread_id": thread_id},
        "metadata": {"user_id": user_id},
    }
    # Extract the last user message
    last_user_text = ""
    for msg in reversed(req.messages):
        if msg.role == "user":
            last_user_text = msg.content or ""
            break
    if not last_user_text:
        yield "Please enter a question."
        return
    # Call LangGraph for streaming output
    async for event in app.astream(
        {"messages": [HumanMessage(content=last_user_text)]},
        config=config,
        stream_mode="values",
    ):
        messages = event.get("messages", [])
        if messages:
            last_msg = messages[-1]
            if isinstance(last_msg, AIMessage) and last_msg.content:
                yield last_msg.content

# Start the server
if __name__ == "__main__":
    server = AgentRunServer(
        invoke_agent=invoke_agent,
        memory_collection_name=os.environ["MEMORY_COLLECTION_NAME"],
    )
    server.start(port=9000)

The following are the header conventions for client requests:

Header

Description

Required

X-AgentRun-Session-ID

Corresponds to LangGraph's thread_id and links multi-turn conversations.

No (If not provided, the system automatically generates a new thread).

X-AgentRun-User-ID

The user ID, which is written to config["metadata"]["user_id"].

No (Defaults to default_user).

To test locally, ensure the port matches the port configuration in s.yaml:

curl -X POST http://localhost:9000/openai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-AgentRun-Session-ID: my-thread-1" \
  -H "X-AgentRun-User-ID: user-1" \
  -d '{"model":"qwen-max","stream":true,"messages":[{"role":"user","content":"Hello"}]}'

Deploy to the AgentRun cloud platform

Install deployment tools

Install Serverless Devs (the s command) via npm:

npm install -g @serverless-devs/s

After installation, verify it:

s --version

Configure your Alibaba Cloud credentials. Run the following command and enter your AccessKey ID and AccessKey Secret as prompted:

s config add
Note

Remember the name you set for the key pair during configuration, such as default. This name is used in the access field of s.yaml.

The s build command requires Docker to build Linux x64-compatible dependency packages. Install Docker Desktop or a compatible environment (such as Podman or Lima) beforehand.

Prepare configuration files

Ensure that the root directory of your project contains a requirements.txt file:

agentrun-sdk[server,langgraph]==0.0.31
python-dotenv>=1.0.0

Create a .fcignore file to exclude sensitive files and the local environment:

.env
.venv

Create the s.yaml configuration file in the project root directory:

edition: 3.0.0
name: my-conversation-service
access: 'default' # Replace with the name of the key pair you configured with 's config add'.
vars:
  region: 'cn-hangzhou'
  agent_name: 'my-langgraph-agent' # Agent name
  role: 'acs:ram::${config("AccountID")}:role/AliyunAgentRunDefaultRole'
  memory_collection_name: 'your-memory-collection-name' # Replace with your memory collection name.
  api_key: 'your-dashscope-api-key' # Replace with your DashScope API key.
resources:
  agent:
    component: agentrun
    props:
      region: ${vars.region}
      agent:
        name: ${vars.agent_name}
        description: 'LangGraph Conversation Service'
        role: ${vars.role}
        code:
          src: ./
          language: python3.12
          command:
            - python3
            - conversation_service_langgraph_server.py
        cpu: 1.0
        memory: 2048
        port: 9000
        instanceConcurrency: 200
        internetAccess: true
        environmentVariables:
          PYTHONPATH: /code/python:/opt/python:/code:/code/python
          MEMORY_COLLECTION_NAME: '${vars.memory_collection_name}'
          DASHSCOPE_API_KEY: '${vars.api_key}'
        logConfig: auto
        endpoints:
          - name: prod
            version: LATEST
            description: 'Production endpoint'

Key configuration fields:

Field

Description

access

The name of the key pair you set with s config add.

vars.role

The RAM role for the agent at runtime. Go to the RAM role management page to view or create it.

code.command

Startup command: Modify the server filename based on the framework you use. The example in this article is conversation_service_langgraph_server.py.

environmentVariables

Environment variables for the cloud runtime that replace the local .env file.

Build and deploy

The cloud environment is Linux x64. Use the s build command to install compatible dependencies in Docker:

s build --use-docker

This command pulls the Function Compute runtime image and runs pip install within it to install dependencies into the python/ directory.

After the build completes, deploy the agent:

s deploy

After a successful deployment, the output includes the agent's access URL:

endpoints:
    -
      name:   prod
      url:    https://<agent-id>.agentrun-data.cn-hangzhou.aliyuncs.com/agent-runtimes/<runtime-id>/endpoints/prod/invocations
      status: READY

Test the deployment

Send a request using the URL from the deployment output. Replace <your-endpoint-url> with the actual URL from the s deploy output:

curl -X POST https://<your-endpoint-url>/openai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-AgentRun-Session-ID: my-thread-1" \
  -H "X-AgentRun-User-ID: user-1" \
  -d '{"model":"qwen-max","stream":true,"messages":[{"role":"user","content":"Hello"}]}'

FAQ

Idempotent initialization

Yes. init_langgraph_tables() is idempotent. It skips creation if the tables or indexes already exist. We recommend calling it at application startup to ensure the required resources are ready.

Relationship between thread_id and session_id

In LangGraph, thread_id is used to distinguish between different conversation threads. When session synchronization is enabled (by setting agent_id), OTSCheckpointSaver maps thread_id to session_id in the conversation table. The two have the same value.

When to set agent_id

It depends on your use case:

  • If not set: Checkpoint data still persists and restores normally, but you cannot query the session list by using list_sessions or search_sessions. This approach is suitable for simple scenarios or for applications that do not require session management.

  • If set (recommended): Each checkpoint write automatically synchronizes a record in the conversation table. This approach is suitable for production applications that need to display or search session lists.

LangGraph vs. LangChain

Dimension

LangGraph (OTSCheckpointSaver)

LangChain (OTSChatMessageHistory)

Persistence granularity

Complete graph state snapshot (includes all channel values).

Single messages (one event per message).

State recovery

Automatically restores all state fields, including messages and custom fields.

Restores only the message list.

Use cases

Complex agent graphs with multiple nodes, conditional branches, and tool calls.

Simple conversational chains (Prompt + LLM).

Initialization method

init_langgraph_tables()

init_langchain_tables()

Supported models

The Conversation Service does not restrict your choice of model. The example uses Qwen via the DashScope API, but you can replace it with any LangChain-supported chat model, such as OpenAI GPT, Anthropic Claude, or Google Gemini. The LLM configuration in LangGraph nodes determines the model choice and is independent of the Conversation Service.

Troubleshooting and common issues

You may encounter the following common issues during integration:

Error scenario

Error message

Solution

Authentication failure

InvalidAccessKeyId or SignatureDoesNotMatch

Run echo $ALIBABA_CLOUD_ACCESS_KEY_ID to check if the environment variable is set. Confirm that the AccessKey is not disabled and that the corresponding RAM user has Tablestore permissions.

Network timeout

ConnectionTimeout or OTSClientError

In the AgentRun console, check the endpoint configuration in the memory store and confirm network reachability. If you run in a VPC, use the internal endpoint. You can also add retry logic at the application layer.

Missing environment variable

KeyError: 'MEMORY_COLLECTION_NAME'

Run echo $MEMORY_COLLECTION_NAME to check if the variable is set. For local development, use a .env file and load it with python-dotenv.

Memory store not found

MemoryCollectionNotFound

Go to the AgentRun console to verify the memory store name, which is case-sensitive. Ensure that it matches the value of the $MEMORY_COLLECTION_NAME environment variable.

Checkpoint recovery failure

The multi-turn conversation fails to restore the historical state.

Verify that multiple calls use the same thread_id. Run echo $MEMORY_COLLECTION_NAME to verify that you are connected to the correct target instance. Check if init_langgraph_tables() successfully created the required tables and indexes on its first run.

We recommend adding exception handling for critical operations in a production environment:

from langgraph.graph import StateGraph

try:
    result = await app.ainvoke(
        {"messages": [HumanMessage(content="Hello")]},
        config=config,
    )
except Exception as e:
    print(f"Graph execution failed: {e}")
    # Check the checkpoint status
    cp = await checkpointer.aget_tuple(config)
    if cp:
        print(f"Last successful checkpoint: {cp.checkpoint['id']}")
    raise