Running multiple asyncio tasks

Mar 25, 2024 ยท 2 min read

When writing async code in Python, it's common to have multiple independent tasks that can run concurrently. The asyncio module provides two handy methods for running async tasks in parallel: asyncio.gather() and asyncio.create_task().

asyncio.gather() for Bundling Tasks

asyncio.gather() runs a collection of async tasks concurrently and blocks until they are all complete:

import asyncio

async def task1():
    print('Task 1 running')
    await asyncio.sleep(1)
    return 'result 1'

async def task2():
    print('Task 2 running') 
    await asyncio.sleep(2)
    return 'result 2'

async def main():
    results = await asyncio.gather(
        task1(),
        task2()
    )
    print(results)

asyncio.run(main())

This runs task1() and task2() asynchronously, waits for both to finish, collects their return values, and assigns them to the results list.

The key benefit of gather() is it blocks the calling code until all bundled tasks complete. This avoids race conditions.

asyncio.create_task() for Unblocked Execution

asyncio.create_task() schedules a task to run but doesn't wait for it:

import asyncio

async def main():
    task1 = asyncio.create_task(task1())
    
    await do_other_work()
    
    await task1

Here task1 starts right away on its own. We can await task1 later if we need its result.

So in summary, gather() groups tasks and waits on them, while create_task() schedules background work to happen anytime.

Using these tools together enables complex concurrent async logic in Python. Just be careful to await any background create_task() calls that you need results from later!

Browse by tags:

Browse by language:

The easiest way to do Web Scraping

Get HTML from any page with a simple API call. We handle proxy rotation, browser identities, automatic retries, CAPTCHAs, JavaScript rendering, etc automatically for you


Try ProxiesAPI for free

curl "http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

<!doctype html>
<html>
<head>
    <title>Example Domain</title>
    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
...

X

Don't leave just yet!

Enter your email below to claim your free API key: