Eliminate pybind11 pure virtual calls: fix C++ to Python override lifetimes with smart_holder and trampoline_self_life_support

Pure virtual calls when C++ invokes Python overrides in pybind11? Fix lifetime issues from shared_ptr using py::smart_holder and trampoline_self_life_support.

When binding C++ abstract base classes to Python with pybind11, it’s easy to fall into a lifetime trap. The symptom is puzzling: you instantiate a Python-derived class, call an overridden method from C++, and end up triggering a pure virtual call on the C++ base. No crash in the binding code, just a runtime error right where you expected the Python override to be invoked.

Minimal example that reproduces the issue

The following code shows a C++ module exposing an abstract Actor and ActorFactory, plus a small orchestrator that creates and invokes the actor. The override in Python exists, yet a pure virtual call happens when C++ calls back.

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include <memory>
#include <string>
#include <iostream>
// Abstract class to be implemented in Python
struct Actor {
    virtual ~Actor() = default;
    virtual void setup() = 0;
};
// Abstract factory to be implemented in Python
struct ActorFactory {
    virtual ~ActorFactory() = default;
    virtual std::shared_ptr<Actor> fetchActor(const std::string& type) = 0;
};
// Uses the factory to obtain an actor and then invokes it
class Orchestrator {
private:
    std::shared_ptr<Actor> actorPtr;
    std::shared_ptr<ActorFactory> makerPtr;
public:
    Orchestrator(std::shared_ptr<ActorFactory> fact) : makerPtr(fact) {}
    void execute(const std::string& name) {
        actorPtr = makerPtr->fetchActor(name);
        actorPtr->setup(); // Pure virtual call happens here instead of Python override
    }
};
// ------------------- Trampolines -------------------
namespace py = pybind11;
struct PyActorShim : public Actor {
    using Actor::Actor;
    void setup() override {
        PYBIND11_OVERRIDE_PURE(
            void,
            Actor,
            setup
        );
    }
};
struct PyActorFactoryShim : public ActorFactory {
    using ActorFactory::ActorFactory;
    std::shared_ptr<Actor> fetchActor(const std::string& type) override {
        PYBIND11_OVERRIDE_PURE(
            std::shared_ptr<Actor>,
            ActorFactory,
            fetchActor,
            type
        );
    }
};
// ------------------- Bindings -------------------
PYBIND11_MODULE(CoreBridge, m) {
    py::class_<Actor, PyActorShim, std::shared_ptr<Actor>>(m, "Actor")
        .def(py::init<>())
        .def("setup", &Actor::setup);
    py::class_<ActorFactory, PyActorFactoryShim, std::shared_ptr<ActorFactory>>(m, "ActorFactory")
        .def(py::init<>())
        .def("fetchActor", &ActorFactory::fetchActor);
    py::class_<Orchestrator>(m, "Orchestrator")
        .def(py::init<std::shared_ptr<ActorFactory>>())
        .def("execute", &Orchestrator::execute);
}

And the Python side that exercises it:

import CoreBridge as cb
class CustomFactory(cb.ActorFactory):
    def __init__(self):
        super().__init__()
        self.creators = {}
    def add_actor(self, key, fn):
        self.creators[key] = fn
    def fetchActor(self, tag):
        return self.creators[tag]()
class CustomActor(cb.Actor):
    def __init__(self):
        super().__init__()
    def setup(self):
        print("Hello, world!")
fac = CustomFactory()
fac.add_actor("ActorA", lambda: CustomActor())
runner = cb.Orchestrator(fac)
runner.execute("ActorA")

What’s actually going on

The root cause is the holder type. With std::shared_ptr as the holder for Python-implemented objects, Python and C++ lifetimes are not tied symmetrically. The Python object keeps the C++ instance alive, but the inverse is not guaranteed. If the Python instance goes away because there are no Python references left, a later call from C++ into what used to be the Python override ends up at the C++ pure virtual function, producing a pure virtual call at runtime.

Fix: py::smart_holder plus trampoline self life support

As of pybind11 v3, py::smart_holder can be used as the holder. It ensures that the Python object remains alive as long as the C++ side holds a reference, aligning lifetimes so that virtual dispatch reaches the Python override. The trampolines should inherit py::trampoline_self_life_support. This combination addresses the lifetime mismatch and preserves correct virtual dispatch from C++ into Python.

Below is the corrected C++ binding using py::smart_holder in class bindings and py::trampoline_self_life_support in trampolines. The orchestrator’s constructor that takes std::shared_ptr remains valid because py::smart_holder can be cast to std::shared_ptr when needed.

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include <memory>
#include <string>
#include <iostream>
struct Actor {
    virtual ~Actor() = default;
    virtual void setup() = 0;
};
struct ActorFactory {
    virtual ~ActorFactory() = default;
    virtual std::shared_ptr<Actor> fetchActor(const std::string& type) = 0;
};
class Orchestrator {
private:
    std::shared_ptr<Actor> actorPtr;
    std::shared_ptr<ActorFactory> makerPtr;
public:
    Orchestrator(std::shared_ptr<ActorFactory> fact) : makerPtr(fact) {}
    void execute(const std::string& name) {
        actorPtr = makerPtr->fetchActor(name);
        actorPtr->setup();
    }
};
namespace py = pybind11;
struct PyActorShim : public Actor, py::trampoline_self_life_support {
    using Actor::Actor;
    void setup() override {
        PYBIND11_OVERRIDE_PURE(
            void,
            Actor,
            setup
        );
    }
};
struct PyActorFactoryShim : public ActorFactory, py::trampoline_self_life_support {
    using ActorFactory::ActorFactory;
    std::shared_ptr<Actor> fetchActor(const std::string& type) override {
        PYBIND11_OVERRIDE_PURE(
            std::shared_ptr<Actor>,
            ActorFactory,
            fetchActor,
            type
        );
    }
};
PYBIND11_MODULE(CoreBridge, m) {
    py::class_<Actor, PyActorShim, py::smart_holder>(m, "Actor")
        .def(py::init<>())
        .def("setup", &Actor::setup);
    py::class_<ActorFactory, PyActorFactoryShim, py::smart_holder>(m, "ActorFactory")
        .def(py::init<>())
        .def("fetchActor", &ActorFactory::fetchActor);
    py::class_<Orchestrator>(m, "Orchestrator")
        .def(py::init<std::shared_ptr<ActorFactory>>())
        .def("execute", &Orchestrator::execute);
}

The Python code that uses the module does not need to change. The override remains the same, and the call from C++ will correctly dispatch into Python without hitting a pure virtual call.

Why this detail matters

Factories and plugin-style architectures are a natural fit for Python-over-C++ patterns, but they also amplify lifetime pitfalls. Objects created in Python and consumed from C++ cross ownership boundaries, so an incorrect holder choice silently decouples lifetimes. That’s how spurious pure virtual calls creep in. Using py::smart_holder and enabling self life support in trampolines keeps both worlds in agreement about object existence, ensuring that virtual dispatch works reliably.

Takeaways

If a Python override of a C++ pure virtual is being called from C++ and you see a pure virtual call instead, scrutinize the holder type. With pybind11 v3, prefer py::smart_holder for classes implemented in Python and called from C++, and make trampoline classes inherit py::trampoline_self_life_support. This aligns lifetimes and preserves correct behavior without changing the higher-level design.