Scraping eBay Listings with Python and BeautifulSoup in 2023

eBay is one of the largest online marketplaces with millions of active listings at any given time. In this tutorial, we'll walk through how to scrape and extract key data from eBay listings using Python and the BeautifulSoup library.

Setup

We'll need the following libraries installed:

requests - for sending HTTP requests to the eBay website

BeautifulSoup - for parsing and extracting data from the HTML of the listings

You can install these using pip:

pip install requests beautifulsoup4

We'll also define the starting eBay URL to scrape and a dictionary of headers to spoof a browser visit:

import requests
from bs4 import BeautifulSoup

url = "<https://www.ebay.com/sch/i.html?_from=R40&_trksid=p2334524.m570.l1313&_nkw=baseball&_sacat=64482&LH_TitleDesc=0>"

headers = {"User-Agent": "Mozilla/5.0..."}

Replace the User-Agent with your own browser's user agent string.

Fetch the Listings Page

We'll use the requests library to fetch the HTML content from the eBay URL:

response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, "lxml")

The headers dictionary is passed to spoof a browser visit. We use the BeautifulSoup parser with the lxml parser to parse the HTML.

Extract Listing Data

Now we can extract the key data points from each listing. eBay encloses each listing in a

with class "s-item__info">. We'll find all of these:

listing_items = soup.find_all("div", class_="s-item__info clearfix")

Then we can loop through each listing div and extract the relevant info using the class names:

for item in listing_items:

  title = item.find("div", class_="s-item__title").text.strip()

  url = item.find("a", class_="s-item__link")["href"]

  price = item.find("span", class_="s-item__price").text.strip()

  # And so on for other fields like seller, shipping, location etc.

We use .text.strip() to extract the text within tags like

and . For the item URL, we extract the href attribute from the tag.

Print Results

Finally, we can print out all the extracted info for each listing:

print("Title:", title)
print("URL:", url)
print("Price:", price)

print("="*50) # Separator between listings

This will output each listing's title, url, price and other data in a readable format.

Full Code

Here is the full code to scrape and extract eBay listing data:

import requests
from bs4 import BeautifulSoup

url = "https://www.ebay.com/sch/i.html?_nkw=baseball"

headers = {"User-Agent": "Mozilla/5.0..."} 

response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, "lxml")

listing_items = soup.find_all("div", class_="s-item__info clearfix")

for item in listing_items:

  title = item.find("div", class_="s-item__title").text.strip()
  
  url = item.find("a", class_="s-item__link")["href"]

  price = item.find("span", class_="s-item__price").text.strip()

  details = item.find("div", class_="s-item__subtitle").text.strip()

  seller_info = item.find("span", class_="s-item__seller-info-text").text.strip()

  shipping_cost = item.find("span", class_="s-item__shipping").text.strip()
  
  location = item.find("span", class_="s-item__location").text.strip()

  sold = item.find("span", class_="s-item__quantity-sold").text.strip()

  print("Title:", title)
  print("URL:", url)
  print("Price:", price)
  print("Details:", details)
  print("Seller:", seller_info)
  print("Shipping:", shipping_cost)
  print("Location:", location)
  print("Sold:", sold)

  print("="*50)

This gives you an overview of how to use Python and BeautifulSoup to extract data from eBay listings. The key steps are sending a request, parsing the HTML, finding the listing divs, and extracting the text from tags based on their class names.

Browse by language:

Scraping eBay Listings with Python and BeautifulSoup in 2023

Setup

Fetch the Listings Page

Extract Listing Data

Print Results

Full Code

Browse by tags:

The easiest way to do Web Scraping

Scraping eBay Listings with Python and BeautifulSoup in 2023

Setup

Fetch the Listings Page

Extract Listing Data

Print Results

Full Code

The easiest way to do Web Scraping

Don't leave just yet!