Making HTTPS Requests in Python with Requests and Certifi

Feb 3, 2024 ยท 2 min read

When making HTTPS requests in Python, it's important to have SSL/TLS certificate verification enabled to ensure secure connections. The requests library makes HTTP requests simple, while certifi provides Mozilla's certificate bundle to verify SSL connections.

Why Verify Certificates?

Verifying SSL certificates prevents man-in-the-middle attacks and ensures you connect to the real server you intend to. Without verification, an attacker could intercept your connection and you'd have no way to detect it. Always verify certificates in production environments.

Using the Requests Library

The Requests library simplifies making HTTP calls in Python:

import requests

response = requests.get('https://example.com')
print(response.text)

However, certificate verification is not enabled by default. To enable it, we'll use Certifi.

Adding SSL Verification with Certifi

Certifi provides Mozilla's curated list of root Certificate Authorities. By telling Requests to use Certifi's CA bundle, certificate verification is enabled:

import requests
import certifi

response = requests.get('https://example.com', verify=certifi.where())

Now all HTTPS requests made using Requests will have their SSL certificates verified against Mozilla's root CA list.

Disabling Verification (Not Recommended)

You can disable verification but should only do this in development environments:

response = requests.get('https://example.com', verify=False)  

Other Tips

  • Always use certificate verification in production.
  • To debug SSL issues, enable Requests' debugging output to see certificate details.
  • For non-public sites using custom certificates, you'll need to provide your custom CA bundle instead of Certifi's.
  • By leveraging Requests and Certifi together, you can easily make verified and trusted HTTPS calls in Python and be confident you have secure connections.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: