Asyncio double-checked locking to ensure a single loader and avoid redundant loads across coroutines

Learn how double-checked locking in asyncio ensures a single loader, prevents duplicate coroutine work, and keeps Redis-backed data refreshes predictable.

Asyncio: ensuring a single loader without duplicate work

When a service watches a Redis-backed timestamp and refreshes in-memory data on change, the goal is straightforward: only one coroutine should perform the load at a time. The snag appears when multiple coroutines race to the same await point and each ends up doing the same work, one after another. The fix is surprisingly small, but important.

Problem setup

The snippet below polls a store for the latest timestamp and, if it differs from the one in memory, triggers a load. The intention is that only one coroutine performs the load while others either wait or use stale data temporarily.

import asyncio

class SyncAgent:
    def __init__(self):
        self.gate = asyncio.Lock()
        self.mark = None

    async def _pull_mark_from_store(self):
        # go to db
        ...

    async def apply_if_needed(self):
        latest = await self._pull_mark_from_store()
        if self.mark == latest:
            return
        if self.gate.locked():
            return  # a load is in progress; OK to use existing data for now
        async with self.gate:
            # do the load
            self.mark = latest

What goes wrong and why

The check for whether the lock is held happens before entering the critical section. Multiple coroutines can observe the lock as free and reach the async with gate line together. They will then enter the critical section one by one and repeat the load. The root cause is the race between the pre-checks and acquiring the lock. The value being compared can change during the pause at the await point, so a single pre-check outside the lock is not enough.

The fix: double-checked locking

The practical pattern here is double-checked locking: perform a quick pre-check to avoid unnecessary locking when nothing changed, then repeat the check after acquiring the lock. This ensures only the first coroutine that sees a change actually performs the load; the rest arrive later, re-check, and see there is nothing left to do.

Moving the check inside the lock is correct.

import asyncio

class SyncAgent:
    def __init__(self):
        self.gate = asyncio.Lock()
        self.mark = None

    async def _pull_mark_from_store(self):
        ...

    async def apply_if_needed(self):
        latest = await self._pull_mark_from_store()
        if self.mark == latest:
            return

        async with self.gate:
            if self.mark == latest:
                return

            # do the load
            self.mark = latest

This preserves the intended behavior. The outside check saves an acquisition when nothing changed. The inside check guarantees correctness when multiple coroutines converge on the same await boundary.

Optional fast-fail when a load is in progress

In some cases, instead of waiting to acquire the lock, it may be preferable to exit immediately if a load is already running. This approach avoids queueing at the lock but still relies on the inner check for correctness. It is suitable with asyncio due to its specific await points for concurrent code, and not for multi-threaded concurrency.

import asyncio

class SyncAgent:
    def __init__(self):
        self.gate = asyncio.Lock()
        self.mark = None

    async def _pull_mark_from_store(self):
        ...

    async def apply_if_needed(self):
        latest = await self._pull_mark_from_store()
        if self.mark == latest:
            return
        if self.gate.locked():
            return  # do not wait; another coroutine is already loading

        async with self.gate:
            if self.mark == latest:
                return

            # do the load
            self.mark = latest

Why this matters

Without the inner check, multiple coroutines can perform the same expensive load. Double-checked locking keeps the code simple while preventing redundant work. It is a common pattern here because state can change between await points, and the second check is what closes the race window.

Conclusion

When coordinating updates across coroutines in asyncio, trust the lock to guard both the work and the verification. Check outside the lock to avoid unnecessary acquisitions, then check again inside the lock to ensure only one coroutine proceeds. If you prefer not to wait behind a lock, returning early when it is already held is a valid strategy in asyncio, but still keep the inner check. This way, you eliminate duplicate loads and keep the refresh path predictable.