Working with JSON Data in Python using urllib

The urllib module in Python provides useful tools for fetching data from the web. When working with APIs, the data is often returned in JSON format. Python has great JSON support, but fetching and parsing JSON with urllib requires a few handy steps.

Fetching JSON Data

To fetch JSON data from a URL with urllib, we can use the urlopen() function:

import urllib.request

with urllib.request.urlopen("https://api.example.com/data") as response:
   json_data = response.read()

This gives us the raw JSON data as a bytes string. We need to decode it to get a Python dict:

import json

json_data = json.loads(json_data)

Now json_data contains our JSON payload as nested Python dicts and lists.

Parsing Nested JSON Data

JSON data can contain complex nested objects. We may only need a small part. Here's how to extract just a list of user names:

names = [user["name"] for user in json_data["users"]]

We can traverse the dict structure to grab what we need.

Handling Errors

APIs can sometimes give errors. We can handle them gracefully:

import urllib.error

try:
   # fetch JSON 
except urllib.error.HTTPError as e:
   print("API error:", e.code, e.reason)   
except urllib.error.URLError:
   print("Connection error")  

This keeps our program running rather than crashing on an API error.

Working with JSON in Python is very common. urllib gives a simple way to fetch and parse web-based JSON APIs. With some basic techniques, we can traverse nested data and stay resilient to errors. Over time, these skills become second nature.