Creating asyncio.Tasks on a background event loop thread in Python without blocking the main thread

Learn how to run an asyncio event loop on a background thread, create real asyncio.Task via run_coroutine_threadsafe, and avoid blocking using asyncio.to_thread.

Running an asyncio event loop on a dedicated background thread is a common pattern when the main thread must remain responsive. The friction starts when you try to add a background job from the main thread and expect an asyncio.Task back, for example to use its name attribute. Submitting a coroutine via asyncio.run_coroutine_threadsafe returns a concurrent.futures.Future, not an asyncio.Task, which breaks code that relies on Task’s API. The solution is to create the Task inside the target loop’s thread and then hand that Task object back to the caller.

Problem setup

The loop lives on a daemon thread and the main thread needs to enqueue work without blocking. Submitting the coroutine directly returns the wrong type for Task-centric logic.

import asyncio
import threading

bg_loop = asyncio.new_event_loop()
threading.Thread(target=bg_loop.run_forever, name="AsyncExecutor", daemon=True).start()

def push(coro):
    fut = asyncio.run_coroutine_threadsafe(coro, bg_loop)
    return fut  # This is concurrent.futures.Future, not asyncio.Task

async def job():
    await asyncio.sleep(1)
    return "Done"

async def main():
    fut = push(job())
    # You can't use Task-specific features here (e.g., .get_name() / name)
    print(fut.result())

asyncio.run(main())

What’s actually going on

asyncio.create_task must be executed inside the event loop that will run the coroutine. When you submit a coroutine from another thread using asyncio.run_coroutine_threadsafe, you get back a concurrent.futures.Future proxy tied to the submission mechanism, not the actual asyncio.Task. That’s by design. If your application logic depends on Task attributes, you need to construct the Task in the loop’s thread and return that Task object to the caller.

Practical fix: create the Task inside the background loop

The pattern below keeps a dedicated loop on a daemon thread, provides a dispatcher to run arbitrary coroutines on that loop, and exposes helpers to create and await tasks while still accessing the actual asyncio.Task instance.

import asyncio
import threading

_bg_loop = asyncio.new_event_loop()
threading.Thread(target=_bg_loop.run_forever, name="AsyncExecutor", daemon=True).start()

def dispatch(coro, yield_result=True):
    """Run a coroutine on the background loop. If yield_result is True,
    block and return the coroutine's result; otherwise return the proxy Future."""
    proxy = asyncio.run_coroutine_threadsafe(coro, _bg_loop)
    return proxy.result() if yield_result else proxy

def spawn_task(coro):
    """Schedule a task on the background loop and return the asyncio.Task."""
    async def make_task():
        return asyncio.create_task(coro)
    return dispatch(make_task())

def wait_task(task, yield_result=True):
    """Await the given task on the background loop. If yield_result is True,
    block and return the task's result; otherwise return a proxy Future."""
    async def watcher():
        return await task
    return dispatch(watcher(), yield_result=yield_result)

if __name__ == "__main__":
    async def do_work():
        await asyncio.sleep(1)
        return "Done"

    async def main():
        t = spawn_task(do_work())
        # Optionally don't block now—get a proxy and resolve it later:
        later = wait_task(t, yield_result=False)
        # ... do other work ...
        print(later.result())

    asyncio.run(main())

This approach returns a real asyncio.Task from spawn_task, so you can access or set its name attribute as needed. The main thread only blocks when you explicitly ask for results via wait_task with yield_result=True or by calling result() on the returned proxy.

Non-blocking handoff using asyncio.to_thread

If you prefer not to block the main thread when waiting for results, delegate the blocking Future.result() call to a worker thread via asyncio.to_thread. All functions become async, and the main coroutine stays responsive.

import asyncio
import threading

_bg_loop = asyncio.new_event_loop()
threading.Thread(target=_bg_loop.run_forever, name="AsyncExecutor", daemon=True).start()

async def dispatch(coro):
    """Run a coroutine on the background loop and await its result."""
    def runner():
        proxy = asyncio.run_coroutine_threadsafe(coro, _bg_loop)
        return proxy.result()
    return await asyncio.to_thread(runner)

async def spawn_task(coro):
    """Schedule a task on the background loop and return the asyncio.Task."""
    async def make_task():
        return asyncio.create_task(coro)
    return await dispatch(make_task())

async def wait_task(task):
    """Await the given task on the background loop and return its result."""
    async def watcher():
        return await task
    return await dispatch(watcher())

if __name__ == "__main__":
    async def do_work():
        await asyncio.sleep(1)
        return "Done"

    async def main():
        t = await spawn_task(do_work())
        # ... do other work ...
        print(await wait_task(t))

    asyncio.run(main())

Why this matters

Background orchestration often relies on Task features like naming, grouping, or tracking lifecycle semantics that don’t exist on concurrent.futures.Future. Returning the real asyncio.Task preserves those semantics and avoids brittle workarounds. At the same time, controlling when and how you block lets you keep the main thread responsive or fully async, depending on your constraints.

Takeaways

If your event loop runs on a separate thread and you need an asyncio.Task back in the caller, create the Task inside that loop. Use a small trampoline coroutine scheduled via asyncio.run_coroutine_threadsafe to call asyncio.create_task and return the Task. If blocking the main thread is a concern, wrap the synchronous wait in asyncio.to_thread so your top-level flow remains async-friendly. With this pattern you keep strict separation of loop ownership, get the right return type, and retain the ability to await results when it suits your control flow.