Encoding URLs with urllib quote

Feb 6, 2024 · 2 min read

When building URLs in Python, it is often necessary to encode special characters to ensure the URL is valid. The urllib.parse.quote() function provides an easy way to accomplish this.

Let's look at a quick example:

from urllib.parse import quote

url = "https://www.example.com/search?term=café"
print(quote(url))

This would output:

https%3A%2F%2Fwww.example.com%2Fsearch%3Fterm%3Dcaf%C3%A9

As you can see, the accented character é has been converted to its percent encoded form %C3%A9. This ensures the URL can be processed correctly.

The quote() function handles encoding all reserved characters defined in RFC 3986. This includes:

  • Non-alphanumeric characters like spaces, braces, asterisks etc.
  • Non-ASCII or accented letters like é or ß
  • Special symbols like ® or €
  • Without proper encoding, these characters could cause issues when used in URLs.

    For example, spaces would get interpreted as a separator rather than part of the term:

    https://www.example.com/search?term=café au lait

    Would become:

    https%3A%2F%2Fwww.example.com%2Fsearch%3Fterm%3Dcaf%C3%A9%20au%20lait

    See how the space got converted to %20?

    The quote() function takes care of all of this automatically. You just pass in the full URL string and it will encode the special characters.

    One thing to note is that quote() will not encode slashes or the question mark in the URL. Those are considered safe and valid characters for URLs.

    In summary, urllib.parse.quote() is extremely useful when constructing URLs in Python. It handles all the intricacies of percent encoding special characters. Be sure to use it if building URLs with non-ASCII text or other symbols that could cause issues. Encoding the parameters ensures that URLs will be processed correctly on the server side.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: