Fix Python logging with QueueListener: make it respect handler levels and keep FileHandler clean

Seeing INFO in a WARNING-only file with Python logging queues? Learn why QueueListener skips handler levels and how respect_handler_level restores proper

When wiring Python logging through a queue, it’s easy to assume that handler-level filtering will behave the same as in a direct handler chain. On Python 3.12.9 on Windows, a setup based on logging.handlers.QueueHandler and logging.handlers.QueueListener can appear to ignore a FileHandler’s setLevel, letting lower-severity records slip into a file. The minimal example below demonstrates how INFO and WARNING both end up in Test.log even though the file handler is set to WARNING.

Reproducing the unexpected output

The code creates a FileHandler configured for WARNING, pushes all log records through a multiprocessing.Queue via QueueHandler, and consumes them with QueueListener. Despite the intended filter, INFO still lands in the file.

import logging
import logging.handlers
from multiprocessing import Queue
sink_file = logging.FileHandler('Test.log', encoding='utf-8')
sink_file.setLevel(logging.WARNING)
sink_file.setFormatter(logging.Formatter('%(levelname)s: %(message)s'))
record_queue = Queue()
listener = logging.handlers.QueueListener(record_queue, sink_file)
listener.start()
root_log = logging.getLogger()
for h in root_log.handlers[:]:
    root_log.removeHandler(h)
queue_sink = logging.handlers.QueueHandler(record_queue)
root_log.addHandler(queue_sink)
root_log.setLevel(logging.DEBUG)
app_log = logging.getLogger("TestLogger")
app_log.info("This is INFO")
app_log.warning("This is WARNING")
listener.stop()
logging.shutdown()

What’s actually happening

The file handler’s level alone isn’t enough when records are funneled through a QueueListener. As written, the listener forwards incoming records to the attached handler regardless of that handler’s own threshold, which is why both INFO and WARNING reach the file. The outcome is surprising if you expect the handler’s setLevel to be enforced automatically in a queued configuration.

The fix

QueueListener supports an option that makes it honor the levels configured on its target handlers. Enabling that option ensures the FileHandler will only emit WARNING and above, while other handlers attached to the same listener can still receive lower-severity traffic if needed. Full details are in the official docs: QueueListener.

import logging
import logging.handlers
from multiprocessing import Queue
sink_file = logging.FileHandler('Test.log', encoding='utf-8')
sink_file.setLevel(logging.WARNING)
sink_file.setFormatter(logging.Formatter('%(levelname)s: %(message)s'))
record_queue = Queue()
listener = logging.handlers.QueueListener(
    record_queue,
    sink_file,
    respect_handler_level=True
)
listener.start()
root_log = logging.getLogger()
for h in root_log.handlers[:]:
    root_log.removeHandler(h)
queue_sink = logging.handlers.QueueHandler(record_queue)
root_log.addHandler(queue_sink)
root_log.setLevel(logging.DEBUG)
app_log = logging.getLogger("TestLogger")
app_log.info("This is INFO")
app_log.warning("This is WARNING")
listener.stop()
logging.shutdown()

Why this matters

Queue-based logging is common when consolidating output from multiple components or processes. It’s also a typical pattern when you want different targets to receive different severities, for example forwarding DEBUG and INFO to one destination while keeping a file clean with only WARNING and above. Without enabling the listener to respect handler-level filtering, the separation blurs and files can accumulate noise.

Takeaways

If you rely on per-handler thresholds with a QueueListener, explicitly enable respect_handler_level so each attached handler enforces its own setLevel. This keeps log files focused, preserves intended filtering across multiple targets, and prevents unexpected records from slipping through in queue-driven pipelines.