Introduction to sessions

Why are sessions needed?

Traditional Function Compute uses a stateless design. Requests are scheduled randomly, and instances scale on demand. This design cannot guarantee that consecutive requests will be routed to the same instance. However, emerging scenarios such as AI Agents strongly depend on state persistence:

• Multi-turn conversations: An agent needs to maintain the context of a multi-turn conversation in memory in real time.

• Environment dependencies: Python libraries that are dynamically installed, temporary models that are downloaded, or intermediate files that are generated during runtime must remain available in subsequent interactions.

Function Compute sessions introduce a state persistence mechanism for functions. This ensures that consecutive requests for a specific session are precisely routed to the same function instance.

Function Compute sessions

A Function Compute session is a persistent binding relationship between a request and a function instance. Within the session's validity period, multiple requests that carry the same session are always routed to the same function instance.

Core session modes: isolated vs. non-isolated

The session feature provides two modes to balance security and cost-effectiveness:

• Session isolation

  • Definition: An instance serves only one session during its lifecycle. The instance is destroyed when the session ends.

  • Core value: Ultimate security and independence. Each agent has a completely isolated CPU, memory, and disk. This mode is suitable for processing private data, executing untrusted code in a sandbox environment, and preventing interference between different users.

• Non-isolated session

  • Definition: Multiple sessions can reuse the same function instance sequentially.

  • Core value: Low latency and high cost-effectiveness. This mode eliminates the need to start a new instance for each new session, which significantly reduces cold starts. It is suitable for scenarios where common dependencies can be shared and isolation requirements are moderate.

Session lifecycle management

The system automatically manages resource costs and user experience using the following two parameters. The session is destroyed when either condition is met:

• To focus on experience: Increase the SessionTTL parameter to ensure the session remains "online" for a long time. Increase the SessionIdleTimeout parameter to prevent the session from being destroyed after a short idle period.

• To focus on cost: Decrease the SessionTTL and SessionIdleTimeout parameters to quickly release inactive resources.

Note: You can define these two parameter values using the session API. For more information, see Session lifecycle management.

Session states

A session goes through the following states during its lifecycle:

1. Running: In this state, the instance is ready and being billed. It includes two sub-states:

   1. Active: The instance is processing invocation requests initiated by InvokeFunction.

   2. Idle (light hibernation): After the request is processed, the instance is kept alive. The memory state and file system are fully preserved, which allows the instance to respond instantly to the next request. In this state, the instance is billed at a lower idle rate. You are charged only for storage fees for resources such as memory and disk, not for CPU fees.

2. Expired: The system automatically terminates the session. A session enters this state when it reaches the SessionTTL limit or its idle time exceeds SessionIdleTimeout. The data is cleared and cannot be recovered.

3. Deleted: The user actively terminates the session. This state indicates that the user manually destroyed the session using the DeleteSession API. This is often used to stop billing immediately after a task is complete.

Affinity protocol types

Function Compute supports multiple mechanisms to identify which requests belong to the same session:

• MCP affinity: This mechanism is compatible with standard MCP Server-Sent Events (SSE)/Streamable modes. You can simply send a request using a standard MCP client. The system automatically intercepts the session identity to achieve zero-configuration affinity.

• HeaderField affinity: This mechanism is highly flexible. You can specify a custom key, such as x-session-id, in the HTTP header. The system uses the key's value for routing and binding. It routes requests with the same session ID to the same instance.

• Cookie affinity: This mechanism is web-friendly and compatible with the standard cookie protocol. For the first request, the Function Compute server-side generates a session ID and includes it in the `Set-Cookie` header of the response. For subsequent requests, the client must include the session ID in the request's `Cookie` header. The system identifies the session ID in the request and routes requests with the same session ID to the same instance.

Concurrency and resource management

• Concurrent sessions per instance: This refers to the number of sessions that can be bound to a single instance at the same time. This value is currently fixed at 1. In non-isolated mode, multiple sessions reuse the instance serially.

• Maximum request concurrency per instance: This prevents instance overload. A single session is limited to 200 concurrent requests. Requests that exceed this limit trigger rate limiting protection.

Dynamic session mounting

• Definition: In session isolation mode, you can dynamically mount storage resources such as Object Storage Service (OSS), Network Attached Storage (NAS), and PolarFileSystem (PolarFS) for specific sessions.

• Core value: Different sessions mount different directories. Using dynamic mounting, you can isolate mount configurations between sessions. This lets you operate on cloud storage within a stateful instance as if you were reading from and writing to a local hard drive.