Background Python Logging with subprocess.Popen: Non-blocking Launch and Graceful Stop via SIGINT

Learn to run a Python logger in the background with subprocess.Popen and stop it cleanly via SIGINT/KeyboardInterrupt, Windows tips and non-blocking control.

Launching a Python logger in the background, letting the parent keep working, and then stopping the logger gracefully is a common control-flow task when you need repeatable data captures. The goal is clear: start a child script with arguments, keep it silent, and when the parent is done, trigger the child’s KeyboardInterrupt handler so it finalizes properly instead of being killed abruptly.

Minimal reproduction of the blocking setup

One of the typical pitfalls is starting the child synchronously, which blocks the parent and defeats the whole idea of background execution. Here is a compact illustration of an approach that will not work for this use case, because it waits for the child to exit before continuing.

import subprocess
import time
log_stub = "capture_001.xlsx"
# This call blocks until the child completes
res = subprocess.run(["python", "logger_child.py", log_stub])
# The code below is never reached while the child is running
# so you cannot drive timers or PLC commands here
 time.sleep(2)

The parent is stuck until the child terminates on its own, which prevents sending a signal at the right time and sequencing your PLC commands.

What actually causes the problem

The control issue stems from using a synchronous API to spawn the child. When you call a blocking function, the parent process cannot schedule timers, talk to your PLC, or decide when to end the logger. For this workflow, the parent must continue immediately after launch and hold a live handle to the child process in order to send a signal later.

Background execution and graceful stop with subprocess

The appropriate tool here is subprocess with Popen. It starts the child process and returns control to the parent right away, giving you a process handle. When you are ready to stop logging, send SIGINT to the child. This mirrors a Ctrl+C and triggers the KeyboardInterrupt handler in the child script, which is precisely the mechanism needed to run its finalization logic before closing the file.

import subprocess
import signal
import time
output_name = "capture_001.xlsx"
child_file = "logger_child.py"
# Start the child logger in the background and keep the handle
child_job = subprocess.Popen(["python", child_file, output_name])
# Parent is free to do PLC work, timers, etc.
# ... start process, run timer 1, send PLC command, wait, send second command ...
# This is just placeholder timing to illustrate sequencing
 time.sleep(5)
# When you are ready to stop the logger, ask it to finalize cleanly
child_job.send_signal(signal.SIGINT)
# Optionally wait for shutdown if you need to coordinate the next run
child_job.wait()

If the child writes to stdout, configure Popen to capture or redirect it so the output does not clutter your console. If it logs to a file, you can leave the streams alone.

Windows specifics you should be aware of

There are limitations to signals on Windows. In particular, sending CTRL_C_EVENT to process groups does not behave like a real keyboard interrupt. This has been observed to be unreliable, and SIGINT via send_signal is the intended mechanism in this setup. For context and a workaround discussion, see the reference: Sending ^C to Python subprocess objects on Windows: https://stackoverflow.com/questions/7085604/sending-c-to-python-subprocess-objects-on-windows.

It also matters that the child has a KeyboardInterrupt handler. If the handler is present in the child, SIGINT will trigger the cleanup path rather than a hard stop.

Why this pattern matters

When automating repeated data collections, you need deterministic boundaries around each run. Starting the logger asynchronously ensures the parent can orchestrate PLC commands and timers precisely. Ending the logger via SIGINT preserves the child’s finalization path so plots, summaries, or file flushing happen reliably. This is the difference between repeatable runs and brittle captures.

Practical wrap-up

Use subprocess.Popen to launch the child logger with a filename argument and keep the returned handle. Drive your timers and PLC commands in the parent while the child runs. When the cycle is done, send SIGINT so the child triggers its KeyboardInterrupt handling and completes its closing actions. If you need the next run to start only after the child has finished writing, wait on the process. On Windows, be mindful of signal limitations and refer to the linked discussion if you run into CTRL_C_EVENT quirks.