When building applications with aiohttp, managing request timeouts properly is key to ensuring good performance. The default timeout settings may not be ideal for all use cases, so understanding how to configure them is important.
The Basics of aiohttp Timeouts
By default, aiohttp has a 5 minute total timeout for a request. This is made up of two separate timeouts:
These can be configured on a per-request basis, or application-wide.
Why the Defaults Can Be a Problem
The 5 minute defaults may be too high for some applications. Problems this can cause include:
Setting the timeouts too low however causes its own issues:
So how do we strike the right balance?
Tuning Timeouts for Your Application
The optimal timeouts depend entirely on your specific application. Here are some things to consider while tuning:
Characterize Normal Request Times
First, analyze request latency under regular load without timeouts kicking in. Set timeouts to be in line with the majority of request times. The defaults are likely too high for most services.
# Timeout slightly longer than 95th percentile latency
client_timeout = 1.5
server_timeout = 2 This prevents unusually slow requests from tying up resources without impacting most requests.
Set Client Timeout < Server Timeout
The client timeout cuts off waits between reads from the server. This should be just long enough to account for server processing time.
The server timeout limits total request processing time on the server. This can be set slightly higher to account for expected variations.
Scale Timeouts Under Load
During periods of high load, latency typically increases. Rather than overwhelming backends with retries, scale timeouts up accordingly.
Conversely, reduce timeouts when systems are underutilized to free up resources quicker.
Configure Timeouts Globally
Rather than configure on every request, use a connector to set application wide policies:
conn = aiohttp.TCPConnector(
timeout=aiohttp.ClientTimeout(total=3))This sets a 3 second total timeout for all requests made with this connector.
Wrapping Up
Finding the right timeout values requires testing and tuning for your specific workflow. Start conservative and increase as needed. Analyze metrics like latency distributions, error rates, and resource usage to fine tune over time. The key is finding the sweet spot that maximizes performance without overloading your services.
Prioritizing request timeouts in your aiohttp application goes a long way towards preventing cascading failures and delivering a smooth user experience.