Fetching News Articles with the Google News API and Python

Feb 3, 2024 ยท 2 min read

The Google News API allows you to programmatically search for and retrieve recent news articles on any topic. This guide will walk through how to use Python to query the API and parse the results.

Getting Set Up

To use the Google News API, you'll need:

  • A Google Cloud account and project with the News API enabled
  • The google-api-python-client package installed
  • First, sign up for a Google Cloud account. Then enable the News API under APIs & Services.

    Next, install the Python client library:

    pip install google-api-python-client

    Finally, generate an API key from your Cloud console and save it - you'll need this later to authenticate requests.

    Constructing a News Query

    The News API uses standard Google query syntax. For example, to search for "python" news:

    query = "python"

    You can refine search results further by adding modifiers like a date range or descriptive keywords:

    query = "new python book released after:2021-01-01 before:2023-01-01"

    See the full docs for other supported parameters.

    Calling the API

    With the API client installed and a query defined, we can now write the Python to fetch articles:

    from googleapiclient.discovery import build
    
    # Construct and send request
    service = build("news", "v2")
    result = service.articles().list(q=query).execute()  
    
    articles = result['articles']

    The returned articles contain all the headlines, descriptions, dates, and more parsed from the query. You can now process the articles as needed - print headlines, save to a database, etc.

    Handling Pagination

    By default the API returns 20 results per page. To handle pagination:

    page_token = None
    while True:
      param = {'q': query, 'pageToken': page_token}  
      res = service.articles().list(**param).execute()
        
      # Your article parsing code  
    
      page_token = res.get('nextPageToken')  
      if not page_token:
        break

    This iterates through all pages while results remain. The full response also includes estimated total matches to help with additional analysis.

    The News API provides an easy way to tap into Google's unmatched news search capabilities. With just a few lines of Python, you can fetch current headlines on any topic.

    Browse by tags:

    Browse by language:

    The easiest way to do Web Scraping

    Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


    Try ProxiesAPI for free

    curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

    <!doctype html>
    <html>
    <head>
        <title>Example Domain</title>
        <meta charset="utf-8" />
        <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
    ...

    X

    Don't leave just yet!

    Enter your email below to claim your free API key: