Timeout parameters
API Gateway relays requests from clients to backend services over TCP connections. Misconfigured timeouts on either side of this relay can cause request failures and, under high load, cascading outages. This topic explains the timeout parameters and the rules for setting them correctly.
Three parameters control TCP connection behavior:
ConnectionTimeout: time allowed to establish a TCP connectionWriteTimeout: time allowed to send a requestReadTimeout: time allowed to read a response from the backend service
Set ConnectionTimeout and WriteTimeout based on your network type. Internet-facing connections typically need larger values (10 seconds is a common baseline for both); internal networks can use smaller values.
Set ReadTimeout based on your backend service's actual processing time. Follow the configuration rules in the next section to avoid connection errors.
Configuration rules
An API call follows this path: the client sends a request to API Gateway, which forwards it to the backend service; the backend processes it and returns a response, which API Gateway forwards back to the client.
Two timeout values govern this path:
ClientReadTimeout: how long the client waits for a response from API Gateway. Configured on the client side.APIGatewayBackendTimeout: how long API Gateway waits for a response from the backend service. Configured in the API Gateway console.
As shown in the diagram:
ClientReadTimeout = T2 + T3 + T4 + T5
APIGatewayBackendTimeout = T2 + T3 + T4
This relationship requires two rules:
APIGatewayReaderTimeoutmust be greater than the business processing time that is required by the backend service.ClientReadTimeoutmust be greater thanAPIGatewayBackendTimeout.
For example, if your backend service handles a request within 10 seconds in most cases, set APIGatewayBackendTimeout to 10 seconds and ClientReadTimeout to 15 seconds. This keeps the TCP connection open long enough for the backend to respond.
If timeouts are set too short, API Gateway cannot find an available TCP connection when the backend takes longer than expected — the connection has already been closed. API Gateway reports an N502RE error, and under high load, this can trigger an avalanche failure. Follow these rules carefully.
Configuration examples
APIGatewayBackendTimeout has a minimum value of 300ms. Values below this threshold are automatically raised to 300ms.
To set ClientReadTimeout, configure it in the client initialization code of your HTTP client connection pool. The following figure shows an example using an API Gateway SDK.
To set APIGatewayBackendTimeout, go to the backend service settings of an API definition in the API Gateway console, as shown in the following figure.
Priority hierarchy of APIGatewayBackendTimeout configuration
APIGatewayBackendTimeout can be set in three locations in the API Gateway console:
Backend services
API backend
Route plugins
When values conflict across these locations, the effective priority is: Route plugin > API backend > backend service.
Use this hierarchy to apply different timeouts at different granularities — for example, set a conservative default at the backend service level, then override it for specific slow routes using a route plugin, without changing the global setting.