Fetching Web Resources with urllib in MicroPython

Feb 6, 2024 ยท 2 min read

The urllib module in MicroPython provides a simple interface for fetching resources from the web. Whether you want to grab some JSON data to parse or download an image, urllib handles the HTTP requests and responses for you.

Making Requests

To make a request, first import urllib:

import urllib

Then open a request to a URL. This returns a urequest object:

response = urllib.request('https://example.com')

You can pass headers in a dictionary and data to send in the request body:

headers = {'User-Agent': 'MicroPython'}
data = {'key': 'value'}
response = urllib.request('https://example.com', headers=headers, data=data)

Handling Responses

The response object has several useful properties:

print(response.status) # status code
print(response.headers) # response headers
print(response.read()) # response body

You can iterate through the response line by line:

for line in response:
    print(line)

Or read a specified number of bytes:

data = response.read(100) # read 100 bytes

Always close the response when finished:

response.close()

Practical Examples

Fetch and parse some JSON:

import urllib.json

response = urllib.request('https://api.example.com')
data = urllib.json.loads(response.read())
response.close()

print(data)

Download an image to a file:

with open('image.jpg', 'wb') as f:
    f.write(urllib.request('https://example.com/image.jpg').read())

So in just a few lines, you can make web requests and handle responses using MicroPython's handy urllib module.

Browse by tags:

Browse by language:

The easiest way to do Web Scraping

Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


Try ProxiesAPI for free

curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

<!doctype html>
<html>
<head>
    <title>Example Domain</title>
    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
...

X

Don't leave just yet!

Enter your email below to claim your free API key: