Fixing "InsecureRequestWarning: Unverified HTTPS Request" in Python

Apr 2, 2024 ยท 3 min read

You may sometimes see an InsecureRequestWarning when making HTTPS requests in Python, telling you:

InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised.

This warning indicates that your Python code is making requests over HTTPS, but without validating the server's SSL certificate. This leaves the connection vulnerable to man-in-the-middle (MITM) attacks.

Let's discuss why you might be seeing this warning and how to properly validate certificates to fix it.

Why Does This Warning Occur?

By default, Python will trust any SSL certificate presented by a server when you make an HTTPS request. This is insecure because it allows the possibility of a bad actor intercepting the request and presenting their own fake certificate.

To prevent this, we need to explicitly enable certificate validation in requests so that Python will verify that the certificate is valid and trusted.

The InsecureRequestWarning is triggered when you have NOT enabled certificate validation, alerting you to this potential security hole.

Enabling Certificate Validation

The easiest way to enable certificate validation is by passing the verify parameter to your request:

import requests

url = "https://example.com"

response = requests.get(url, verify=True) 

Setting verify=True will make requests validate SSL certificates for you automatically using trusted certificate authorities. This is the recommended approach.

Alternatively, you can pass the path to a custom CA bundle file containing certificates:

response = requests.get(url, verify="path/to/ca/bundle.pem")

If you need more customization around certificate validation, you can pass an SSLContext object instead:

import ssl

context = ssl.create_default_context(cafile="path/to/ca/bundle.pem")
response = requests.get(url, verify=context)

Troubleshooting Issues

Sometimes you may still run into SSLError exceptions even after enabling certificate validation:

SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed

This typically means that the server is presenting a certificate that cannot validated against the root certificates.

There are a few potential causes:

  • The server could be using a self-signed certificate instead of one signed by a trusted CA. You'll need to explicitly add this certificate as trusted if you want to allow the connection.
  • The server certificate has expired or is invalid. The server needs to renew their SSL certificate.
  • There are issues accessing CA certificate files due to permission problems. Check that Python can read the certificate file paths.
  • The CA that issued the website's certificate is not trusted by your Python installation. You may need to update the trusted CA bundle.
  • Depending on the circumstances, you have a few options to resolve these problems:

  • Add the server's certificate as explicitly trusted by passing an SSLContext.
  • Update the SSL certificate bundle to include the missing CA.
  • Allow the insecure request to proceed if absolutely needed by passing verify=False. This disables validation and will hide the warning, but leaves you vulnerable to MITM attacks.
  • Key Takeaways

  • Use verify=True whenever possible to validate certificates automatically.
  • Pass custom CA bundles or SSLContext objects if you need more control.
  • Troubleshoot SSLError issues by checking server certificates, file permissions, and your Python CA trust bundles.
  • Only set verify=False as an absolute last resort since it disables critical security checks.
  • Enabling SSL certificate verification helps protect your Python applications from attacks. Taking the time to properly handle certificates will give you more secure HTTPS connections.

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: