Parsing XML with BeautifulSoup

While BeautifulSoup is mainly designed for parsing HTML, it can also handle XML documents quite well with just a little configuration. Here's how to leverage BeautifulSoup for scraping and analyzing XML files or responses.

Loading the XML

Loading an XML document into a BeautifulSoup object is the same process as with HTML:

from bs4 import BeautifulSoup

with open("file.xml") as f:
  data = f.read()

soup = BeautifulSoup(data, "xml")

Notice here we explicitly tell it to use the "xml" parser.

Navigating the Tree

You can navigate and search the parsed XML tree using the same methods as HTML:

titles = soup.find_all("title")

first_title = titles[0]
print(first_title.text)

The tag and attribute names will match those defined in the XML.

Finding by Attributes

Searching by attributes works the same:

songs = soup.find_all("song", {"length": "short"})

This finds all tags with a "length" attribute of "short".

Modifying the Tree

You can also modify and add to the XML tree:

new_tag = soup.new_tag("priority")
new_tag.string = "urgent"

first_title.append(new_tag)

This adds a new tag to the first