Inspecting Requests in Python with the Requests Library

Feb 3, 2024 ยท 2 min read

The Python Requests library makes sending HTTP requests simple. But once you've sent a request, how can you see exactly what data was transmitted? There are a couple handy methods on request objects that let you inspect both the request headers and body that were sent.

Accessing Request Headers

After making a request, the request.headers attribute contains a dictionary of all headers that were sent:

import requests

resp = requests.get('https://example.com')
print(resp.request.headers)

This includes things like User-Agent, Accept, Connection, etc.

Viewing Request Body Data

For requests that send data in the body, like POST, you can access it with request.body:

data = {'key': 'value'}
resp = requests.post('https://api.example.com', data=data)

print(resp.request.body)
# b'key=value'

However, this will show the urlencoded data, which may not be very readable. To get a nice string, you can set the json parameter instead of data:

import json

data = {'key': 'value'}
resp = requests.post('https://api.example.com', json=data) 

print(resp.request.body)
# {"key": "value"} 

Now the printed body contains the JSON string that was sent in the request.

Summary

  • Use request.headers to view the headers sent in a Requests request
  • Access the request body with request.body
  • Set json instead of data parameter to have readable body printed
  • Inspecting the request headers and body can be helpful in debugging issues with APIs or just to confirm what data your Python code is transmitting. The Requests library makes it straightforward to see all aspects of HTTP requests.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: