Safely mixing asyncio with synchronous callbacks: using run_coroutine_threadsafe for WeasyPrint and aiohttp

Learn how to bridge asyncio and synchronous libraries with run_coroutine_threadsafe: keep the event loop responsive while integrating WeasyPrint and aiohttp.

Bridging asyncio with a strictly synchronous third‑party library is a classic friction point: your app runs on an event loop, you rely on aiohttp and its middlewares, but a library like weasyprint expects a sync callback such as a url fetcher. You want to reuse existing async logic rather than maintain a parallel synchronous code path. The moment you try to glue them together, you hit errors about loops, threads, and who owns what.

Where it goes wrong

Consider an approach that pushes the blocking library into a worker thread and then tries to call an async coroutine from a sync callback via loop.run_until_complete. The intent is straightforward, but the event loop is already running, so re‑entering it crashes the party.

import asyncio
import io
import aiohttp
from weasyprint import HTML
async def render_pdf_weasy(html: io.IOBase, session: aiohttp.ClientSession):
    loop = asyncio.get_running_loop()
    resolver = _compose_fetcher(session, loop)
    # Run blocking rendering off the event loop:
    return await asyncio.to_thread(_render_pdf_sync, resolver, html)
def _render_pdf_sync(resolver, html: io.IOBase):
    sink = io.BytesIO()
    HTML(file_obj=html, url_fetcher=resolver).write_pdf(sink)
    return sink
def _compose_fetcher(session: aiohttp.ClientSession, loop: asyncio.AbstractEventLoop):
    def fetch(url, *args, **kwargs):
        # This will fail: the loop is already running.
        data, mime = loop.run_until_complete(_pull_bytes(session, url))
        return {"string": data, "mime_type": mime}
    return fetch
async def _pull_bytes(session: aiohttp.ClientSession, url: str):
    async with session.get(url) as resp:
        body = await resp.content.read()
        return body, resp.content_type

This mirrors a common failure mode. Even if the blocking library runs in a worker thread, the callback executes there too, and loop.run_until_complete targets the main loop that is already running. In a different attempt, you might try to create a second loop in another thread and shuttle values over a queue. That collides with another constraint: objects like an aiohttp.ClientSession are tied to the loop where they were created and can’t hop to a different loop without issues. The result is a range of errors about the wrong event loop and a running loop that refuses re‑entry.

The core of the problem

Your async function calls a sync API that takes a sync callback, but you need the logic inside the callback to be async. Calling the sync API directly from the coroutine blocks the event loop; moving it to a worker thread fixes that part, but now the callback runs off the event loop and cannot directly await. Reusing loop.run_until_complete from that thread does not work because the main loop is already running.

What you need is a safe way for a non‑event‑loop thread to schedule an async coroutine back onto the main loop and synchronously wait for its result in that thread. That is exactly what asyncio.run_coroutine_threadsafe provides.

“Can you try run_coroutine_threadsafe”

The pattern that works

Keep the blocking call off the loop using asyncio.to_thread (or asyncio.run_in_executor). Capture the main event loop, pass it into the worker thread, and when the sync callback is invoked, schedule the real async logic onto that loop with asyncio.run_coroutine_threadsafe and wait for the result. The event loop keeps spinning; the worker thread blocks until the coroutine completes; the callback remains synchronous from the library’s perspective.

import asyncio
from collections.abc import Callable
from functools import partial
async def launch_async_entrypoint() -> None:
    payload: int = 1
    # Hand off the blocking sync call to a thread to avoid blocking the loop:
    print('api call argument:', payload)
    outcome = await asyncio.to_thread(blocking_bridge, asyncio.get_running_loop(), payload)
    print('api call result:', outcome)
def blocking_bridge(loop: asyncio.AbstractEventLoop, payload: int) -> int:
    # Call the sync API and provide a sync callback that can hop back to the loop:
    return pretend_api(payload, partial(bridge_callback, loop))
def bridge_callback(loop: asyncio.AbstractEventLoop, value: int) -> None:
    fut = asyncio.run_coroutine_threadsafe(async_handler(value), loop)
    fut.result()  # Wait until the async callback completes
async def async_handler(payload: int) -> None:
    print('callback argument:', payload)
    # ... async work here
def pretend_api(x: int, cb: Callable[[int], None]) -> int:
    y = x * 2
    cb(y)
    return y
if __name__ == '__main__':
    asyncio.run(launch_async_entrypoint())

This prints exactly what you want: the event loop stays responsive, the sync API executes in a separate thread, the sync callback pushes work back to the loop safely, and the final result is returned to the async caller.

How this maps to weasyprint and aiohttp

In a setup with weasyprint as the blocking library and aiohttp providing the network client, keep weasyprint in a thread using asyncio.to_thread. Build a sync url_fetcher that captures the main event loop, and from inside that fetcher call asyncio.run_coroutine_threadsafe on your existing async fetch routine that uses the aiohttp session you already have. That preserves your single source of truth, including middlewares, and avoids maintaining a duplicate synchronous HTTP stack.

Why it matters

Mixing sync and async in Python is less about raw performance and more about control flow correctness. The event loop must never be blocked by long synchronous work. Objects bound to a loop cannot be reused in arbitrary loops or threads. The safe handoff between threads and the event loop is crucial: without it you either deadlock, reenter a running loop, or violate loop affinity of async resources. This pattern solves exactly that junction without forcing you to rewrite your networking as a sync client.

Takeaways

Run the blocking library in a worker thread, keep the main event loop clean, and use asyncio.run_coroutine_threadsafe from the thread to schedule any required async callback logic back onto the loop. If a library ever provides a native async callback variant, prefer it; until then, this approach keeps your code cohesive and your event loop healthy.