Handling Cross-Origin Requests in Python with CORS

Feb 3, 2024 ยท 2 min read

When building web applications, you may need to make HTTP requests from your Python code to APIs on different domains than your own. This is known as a cross-origin request. By default, browsers block these requests for security reasons in a policy known as same-origin.

However, servers can explicitly allow cross-origin requests using CORS (Cross-Origin Resource Sharing). To take advantage of this, we need to properly configure both the client and server sides.

On the client-side, Python's requests module sets certain CORS headers by default, but handles validation and errors for you. So making cross-origin requests in Python code is very simple:

import requests

response = requests.get('https://api.example.com/data')

The key thing to understand is that strict-origin-when-cross-origin is one of the security policies that applies to CORS requests.

What does "strict-origin-when-cross-origin" mean?

This policy states that whenever a cross-origin request is made, the server must check that the Origin header exactly matches the source domain making the request according to the same-origin policy. This prevents malicious sites from spoofing requests.

So if your Python client tries to access https://api.example.com from https://www.my site.com, the Origin header would be set to https://www.my site.com. The API server checks if this matches, and decides whether to allow the CORS request.

Practical Challenges

  • Servers must whitelist allowed origins, rather than allowing all domains.
  • Session cookies may not be sent by browsers in CORS requests, requiring alternatives like JSON web tokens.
  • Errors may not be readable by the client if server doesn't send CORS headers.
  • To handle the nuances of CORS in production systems, it's best to use a dedicated Python package like flask-cors. But understanding the core mechanisms helps debug issues when they do arise!

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: