Controlling Redirections in Python Requests

Feb 3, 2024 ยท 2 min read

When making HTTP requests with the Python Requests library, you may encounter redirections where the server responds with a 3xx redirect status code telling your client to fetch the content from a different URL.

By default, Requests will automatically follow redirections, but you can override this behavior. Here's how to disable automatic redirects in Requests:

import requests

response = requests.get('http://example.com', allow_redirects=False)

Setting allow_redirects=False prevents Requests from automatically following any redirect responses.

This means if the server responds with a 302 or 301 redirect pointing to a different location, Requests will simply return that response object to you rather than fetching the resource from the new location.

Some cases where you may want to disable redirects:

  • You want to examine the redirect response headers/metadata before deciding whether to follow it
  • You want to limit requests to only a certain domain or set of URLs
  • You are rate limited and don't want Requests automatically making additional requests
  • When working with redirects, you can check the response status code to see if a redirect occurred:

    if response.status_code in (301, 302):
        # Redirect occurred

    And to then manually follow the redirect location specified in the Location header:

    location = response.headers['Location']  
    response = requests.get(location)

    Controlling redirects gives you more visibility into exactly what requests are made. Disable auto redirects in Python Requests using allow_redirects=False whenever you want to handle redirects manually.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: