Persisting Cookies from Initial Request in Python Requests

When making requests in Python using the requests module, you may want to save and re-use cookies from an initial request in subsequent requests. This allows you to maintain session state and authentication across requests.

Here are some ways to accomplish this:

Save Cookies to Variable

The simplest approach is to save the cookies from the response to a variable and add them to future requests:

import requests

# Initial request to get cookies
response = requests.get('https://example.com') 

# Save cookies to variable
cookies = response.cookies

# Subsequent requests use cookies
response = requests.get('https://example.com/user', cookies=cookies)

This saves all cookies to a Python RequestsCookieJar object which can be passed directly to future requests.

Use a Session

A better way is to use a Session object, which will automatically handle persisting cookies across all requests:

import requests

session = requests.Session()

# Initial request saves cookies to session
response = session.get('https://example.com')  

# Subsequent requests will use those cookies automatically
response = session.get('https://example.com/user') 

This approach is recommended as it avoids manually handling the cookie persistence. The session handles that for you.

Potential Gotchas

A couple things to watch out for when working with cookies:

Summary

Key points:

Using sessions is the most robust way to ensure cookies are available to all your script's requests. This allows you to maintain state as a client accesses a cookie-based web application.