May 5th, 2020
Is It Legal to Scrape Facebook Data

Let's check the Facebook.com robots.txt file.

When I open the URL on my browser is says.

Ouch. The very first line declares that it is prohibited to scrape Facebook without the express permission of Facebook.

They allow specific bots to crawl them at particular endpoints.

User-agent: Googlebot
Allow: /ajax/bootloader-endpoint/
Allow: /ajax/pagelet/generic.php/PagePostsSectionPagelet
Allow: /safetycheck/

But for the rest of the crawlers, the sign is, stop, turn, and go back to where you came from.

User-agent: *
Disallow: /

So unless you are Cambridge Analytica, you may want to proceed with caution :-)

What about GDPR

GDPR applies to users from the EU, and it merely makes it tricky to scrape data about an individual's data. Any personally identifiable information when scraped makes it automagically illegal in the EU.

What about Accessing data through the Facebook API

You are free to download and use the API data. Still, Facebook has made it increasingly difficult to access a large a=part of its data after the increased scrutiny of their privacy practices after the Cambridge Analytica scandal.

Share this article:

Get our articles in your inbox

Dont miss our best tips/tricks/tutorials about Web Scraping
Only great content, we don’t share your email with third parties.
Icon