Fixing shared state in Python multiprocessing.Manager: why mutation propagates but rebinding doesn't (and how to replace lists in place)

Learn why Python multiprocessing.Manager propagates mutations to managed lists but not rebinding. See proxy objects and how slice assignment replaces contents.

Passing state between Python processes looks straightforward until you try to replace shared containers wholesale. With multiprocessing.Manager, appending to a shared list shows up everywhere, but rebinding the variable to a brand-new list does not. The difference is subtle yet critical: mutation propagates, rebinding does not.

Problem setup

The following minimal example demonstrates the discrepancy. Extending the managed list is visible across processes, whereas assigning a new list to the same variable isn’t reflected elsewhere.

from time import sleep
from multiprocessing import Manager, Process
def writer_task(shared_seq):
    sleep(1)
    shared_seq.extend(["a", "b", "c"])  # propagated
    # shared_seq = ["a", "b", "c"]  # not propagated
def reader_task(shared_seq):
    step = 0
    while step < 8:
        step += 1
        print(f"{step}: {shared_seq}")
        sleep(0.2)
def run_demo():
    with Manager() as mgr:
        shared_seq = mgr.list()
        proc = Process(target=writer_task, args=(shared_seq,))
        procs = [proc]
        proc.start()
        proc = Process(target=reader_task, args=(shared_seq,))
        procs.append(proc)
        proc.start()
        for proc in procs:
            proc.join()
        print("---")
        print(list(shared_seq))
if __name__ == "__main__":
    run_demo()

What’s actually happening and why

When you write shared_seq = ["a", "b", "c"], you aren’t modifying the managed list; you’re only rebinding the local variable shared_seq to a completely new, regular Python list. Nothing about the managed object itself changed, so there is nothing to propagate.

In contrast, calling methods like .append or .extend mutates the managed list object. Managed containers provided by multiprocessing.Manager forward such mutations across process boundaries. This aligns with the rule of thumb highlighted in the Python docs around proxy objects and echoed by practitioners: perform operations that mutate the managed object, not variable reassignments. Assigning to a variable never mutates. An important nuance is that item assignment, like data[key] = value, does mutate because under the hood it dispatches to a method call (data.__setitem__(key, value)).

A brief reference to the official note on this behavior is here: proxy objects. The practical question then becomes “what counts as mutation?” Methods that change the object’s contents do; simple rebinding does not.

Fixing the code

If you really want to replace the contents of a managed list in place, assign to its slice so the list object itself is mutated rather than replaced. This preserves the proxy and propagates the change:

import multiprocessing as mp
def in_place_overwrite(proxy_list):
    proxy_list[:] = ["x", "y", "z"]
if __name__ == "__main__":
    with mp.Manager() as mgr:
        args = [mgr.list("abc")]
        print(*args)
        job = mp.Process(target=in_place_overwrite, args=args)
        job.start()
        job.join()
        print(*args)

This produces the expected transition from ["a", "b", "c"] to ["x", "y", "z"]. The key is that proxy_list[:] = ... mutates the existing managed list rather than rebinding the name proxy_list.

Why this matters

Parallel code is already hard enough; invisible no-ops make it worse. Misunderstanding the difference between mutating a managed container and rebinding a local variable leads to elusive bugs and inconsistent state across processes. The same principle applies to shared dictionaries: use mutating operations, including item assignment, to change the managed object.

Takeaways

When sharing lists or dicts via multiprocessing.Manager, focus on operations that mutate the managed object. Method calls like .append, .extend, and item assignments propagate. Simple assignment to a variable creates a new regular object and leaves the managed one untouched. When you need a full content replacement, use in-place techniques such as slice assignment for lists. Keeping this distinction clear will save time, avoid race-condition rabbit holes, and make your parallel code dependable.