Making HTTP Requests in Python with HTTPX

Feb 5, 2024 ยท 2 min read

Python has several popular libraries for making HTTP requests like requests and urllib. A relatively newer option is HTTPX. In this article, we'll learn how to use HTTPX to make HTTP requests in Python.

What is HTTPX?

HTTPX is a next generation HTTP client for Python, built for making API requests simple and reliable. Some key features of HTTPX include:

  • HTTP/1.1 and HTTP/2 support
  • Sync and async APIs
  • Connection pooling
  • HTTP proxies
  • Timeout configuration
  • Automatic retries
  • Installation

    Install HTTPX with pip:

    pip install httpx

    Making a GET Request

    Here is a simple example to make a GET request and print the response body:

    import httpx
    
    with httpx.Client() as client:
        r = client.get("https://api.example.com/users")
        print(r.text)

    The client handles opening and closing connections automatically.

    POST Requests

    Making a POST request is just as simple. Pass the data to json to send a JSON body:

    data = {"name": "John Doe", "email": "[email protected]"}
    
    with httpx.Client() as client:
        r = client.post("https://api.example.com/users", json=data)

    Handling Responses

    HTTPX responses contain properties like status_code, headers,content and more:

    r = client.get("https://api.example.com/status")
    
    print(r.status_code) # 200
    print(r.headers["Content-Type"]) # "application/json" 
    print(r.text) # Print response body

    Check responses for errors:

    if r.status_code == 200:
        print("Success!")
    else:
        print("Request failed with status", r.status_code)  

    Timeouts, Retries and More

    Some other useful configuration options include:

    client = httpx.Client(
        timeout=5.0, # Timeout per request
        retries=2, # Automatic retries
        verify=False, # Disable SSL verification
    )

    There are many more options for customizing the client and handling advanced use cases.

    Conclusion

    HTTPX makes sending HTTP requests in Python very simple, while providing useful functionality like connection pooling, support for HTTP standards, and configurable retries. Key takeaways:

  • Install with pip install httpx
  • API mirrors Python's requests for ease of use
  • Sync and async support
  • Feature rich client with timeouts, retries and more
  • Handles HTTP/1.1 and HTTP/2 automatically
  • Check out the HTTPX documentation to learn more!

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: