Real-Time Matplotlib Streaming: Correct datetime/date2num Usage and Fast set_data Updates

Learn how to build a real-time Matplotlib plot in Python: stream data, fix timestamp/date2num issues, and update a single line with set_data for fast refresh.

Streaming data into a Matplotlib chart sounds straightforward until you try to keep the plot live, efficient and correctly timed. Two pitfalls appear almost immediately: updating the figure the wrong way inside a tight loop, and feeding the axis with timestamps Matplotlib doesn’t understand. Below is a practical walkthrough that shows what goes wrong and how to fix it cleanly.

Test setup that reveals the issue

The snippet below simulates a network producer pushing values and attempts to plot them against the current time on a fixed-width window. It highlights both problems: the update strategy inside the loop and the way time is converted to Matplotlib’s date format.

import threading
import random
import time
import signal
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as md

class Buf2D:
    def __init__(self, capacity):
        self.grid = np.zeros((capacity, 2))  # rows x 2 columns
        self.capacity = capacity
        self.idx = 0
    def __len__(self):
        return len(self.grid)
    def __str__(self):
        return str(self.grid)
    def append_row(self, row):
        self.grid[self.idx] = row
        self.idx = (self.idx + 1) % self.capacity
    def span_x(self):
        return (self.grid.min(axis=0)[0], self.grid.max(axis=0)[0])

class Feeder(threading.Thread):
    def __init__(self):
        super().__init__()
        random.seed()
        self.alive = True
        self.store = Buf2D(100)
    def payload(self):
        return self.store.grid
    def shutdown(self):
        self.alive = False
    def run(self):
        while self.alive:
            now_ms_num = md.date2num(int(time.time() * 1000))  # ms
            point = np.array([now_ms_num, np.random.randint(0, 999)])
            self.store.append_row(point)
            time.sleep(0.1)

app_active = True

def on_signal(sig, frame):
    global app_active
    app_active = False


def launch():
    signal.signal(signal.SIGINT, on_signal)
    prod = Feeder()
    prod.start()

    fig, axis = plt.subplots()
    formatter = md.DateFormatter('%H:%M:%S.%f')
    axis.xaxis.set_major_formatter(formatter)
    plt.show(block=False)

    while app_active:
        xr = prod.store.span_x()
        axis.set_xlim(xr)
        print(prod.payload())
        plt.draw()
        plt.pause(0.05)

    prod.shutdown()

What’s actually wrong

The first trap is update strategy. Continuously calling plotting functions like plot inside the loop adds new artists every cycle, which is slow and quickly turns the figure into a mess of overlapping lines. The efficient pattern is to create a single Line2D once and only update its data via set_data inside the loop. To refresh the screen interactively, stick to fig.canvas.draw_idle() or simply plt.pause() in the loop; the latter is the most straightforward path for live updates.

The second trap is time. Matplotlib’s dates are floats measured in “days since 0001-01-01 UTC”. Feeding it raw millisecond integers, or converting millisecond integers via date2num, won’t work as intended and leads to formatter errors such as OverflowError: int too big to convert. One observation is that if everything passed into an array is explicitly an int, NumPy preserves that integer dtype; the formatter then still sees massive ints. If a millisecond-scale value is needed as a float, multiplying by 1e3 creates a float. However, the most reliable approach here is to use datetime.datetime.now() and md.date2num() to produce the proper day-based float that Matplotlib expects. Millisecond labels aren’t directly supported out of the box, but you can format ticks to show fractional seconds with a formatter like %H:%M:%S.%f.

Clean fix for smooth, real-time updates

The corrected version below keeps a single line object, updates it with fresh data, resizes the time window to a fixed horizon ending at “now”, and refreshes the canvas in an interactive loop. Time values are stored as Matplotlib dates derived from datetime.

import threading
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as md
import datetime
import time
import signal

class RingStore:
    def __init__(self, limit):
        self.buf = np.zeros((limit, 2))
        self.limit = limit
        self.pos = 0
        self.wrapped = False

    def push(self, row):
        self.buf[self.pos] = row
        self.pos = (self.pos + 1) % self.limit
        if self.pos == 0:
            self.wrapped = True

    def view(self):
        if self.wrapped:
            return np.vstack((self.buf[self.pos:], self.buf[:self.pos]))
        return self.buf[:self.pos]

    def x_window(self, window_seconds=10):
        current = self.view()
        if len(current) == 0:
            now = md.date2num(datetime.datetime.now())
            return (now - window_seconds / 86400.0, now)
        last = current[-1, 0]
        return (last - window_seconds / 86400.0, last)

class Streamer(threading.Thread):
    def __init__(self):
        super().__init__()
        self.on = True
        self.store = RingStore(1000)

    def stop(self):
        self.on = False

    def run(self):
        while self.on:
            ts = md.date2num(datetime.datetime.now())
            val = np.random.randint(0, 999)
            self.store.push(np.array([ts, val]))
            time.sleep(0.1)

running = True

def trap(sig, frame):
    global running
    running = False


def start():
    signal.signal(signal.SIGINT, trap)
    src = Streamer()
    src.start()

    fig, ax = plt.subplots()
    fmt = md.DateFormatter('%H:%M:%S.%f')
    ax.xaxis.set_major_formatter(fmt)
    trace, = ax.plot([], [], 'b-')
    ax.set_ylim(0, 999)
    plt.show(block=False)

    while running:
        current = src.store.view()
        if len(current) > 0:
            trace.set_data(current[:, 0], current[:, 1])
            ax.set_xlim(src.store.x_window(window_seconds=10))
            ax.figure.canvas.draw_idle()
        plt.pause(0.05)

    src.stop()

Why this matters

Real-time visualization is unforgiving to inefficient redraws and malformed time data. Creating new plot artists in a loop silently degrades performance and warps memory usage, while incorrectly scaled timestamps break formatters and axis logic. Adopting a single updatable line and converting time to Matplotlib’s native day-based float ensures that your chart stays responsive and your time axis remains accurate. Formatting ticks with fractional seconds gives you the millisecond-level readability you want without fighting the underlying date model.

Takeaways

When building a live Matplotlib chart backed by a background producer, keep the rendering path simple: instantiate one line, update its data, and let plt.pause() or a canvas draw trigger keep the UI responsive. Treat time with care by generating timestamps via datetime.datetime.now() and md.date2num(), then frame your viewport over a fixed trailing window. This small discipline avoids duplicate artists, prevents overflow in formatters, and yields a smooth, stable plot that’s ready for production traffic.