When working with the popular Python Requests library for making HTTP requests, it can be useful to enable more verbose debug logging to see exactly what is happening under the hood. By default, Requests doesn't log much, but with a few tweaks you can get insightful debug output.
Why Enable Debug Logging
There are a few great reasons to turn on debug logging in Requests:
So if you need more visibility into your Python HTTP requests, debug logging is there for you.
Quick Example
Enabling debug logging in Requests just takes a few lines of code:
import logging
import requests
# These two lines enable debugging at httplib level (requests->urllib3->http.client)
# You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
# The only thing missing will be the response.body which is not logged.
try:
import http.client as http_client
except ImportError:
# Python 2
import httplib as http_client
http_client.HTTPConnection.debuglevel = 1
logging.basicConfig() # you need to initialize logging, otherwise you will not see anything from requests
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
requests.get('http://httpbin.org/headers')This uses the logging module to tap into the debug messages of both Requests and the underlying urllib3 HTTP client.
The key things it does:
Now you'll see verbose logs on the console for the full request and response flow!
Conclusion
Debug logging in Requests provides much deeper insight and can save hours of headache debugging HTTP issues. Give it a try next time you need to peek under the hood!