How to Clear the Cache in Python Requests

Feb 1, 2024 ยท 2 min read

When making HTTP requests with the Python Requests library, it utilizes caching for performance. This saves time by avoiding repeat network requests for the same resource. However, sometimes you need to clear the cache to fetch fresh data or troubleshoot issues. Here are a few ways to clear the Requests cache in Python.

Use session.close()

The easiest way is closing the current session. This clears all cached connections and cookies:

import requests

session = requests.Session()
response = session.get('https://example.com') 

# Clear cache
session.close()  

Creating a new Session instance also works.

Manually Clear the Cache

You can manually clear the cache too by setting the cache attribute on a Session to None:

import requests

session = requests.Session()

# Clear cache
session.cache = None

This clears the cache while reusing the same Session.

Use Cache Control Headers

Another approach is using the "Cache-Control" request header to avoid caching:

import requests

url = 'https://api.example.com/data'

headers = {'Cache-Control': 'no-cache'}

response = requests.get(url, headers=headers)

The "no-cache" directive tells web servers and proxies to fetch fresh data.

Things to consider

  • Clearing the cache for every request causes overhead. Balance performance vs freshness requirements.
  • Server responses may still get cached by proxies and browsers. The above clears the Python Requests cache only.
  • Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: