Build reliable wheels for CPython C++ extensions with setuptools by passing project root and normalizing include/library paths

Learn to package a CPython extension wrapping a C++ library with setuptools and PEP 517: pass the project root, fix include_dirs/library_dirs, build wheels.

Packaging a CPython extension that wraps an existing C++ library sounds straightforward until the build backend relocates your sources into a temporary directory and all your carefully crafted relative paths to headers and .a/.so files stop resolving. If you are using Python’s C API directly, building with setuptools, and want to keep your C++ project as-is—no Boost.Python, no Cython, no new repo—this guide walks through a practical approach to make setuptools aware of the original project location during the wheel build.

Context and the failing scenario

The project keeps a C++ library and a Python wrapper together, built by GNU make for the C++ part and setuptools for the Python extension. The repository layout looks like this:

.
├── include
│   └── …
├── Makefile
├── python
│   ├── wrapper-module
│   │   └── …
│   ├── pyproject.toml
│   ├── setup.py
│   └── …
├── src
│   └── …
└── tests
    └── …

Invoking the build with python -m build causes the backend to copy the build inputs into an isolated location. Even with --no-isolation, setuptools builds from a separate directory. As a result, any relative include-dirs and library-dirs that point back to the C++ library’s build outputs or headers can no longer be resolved. Attempts to hack around this—copying artifacts via Makefile steps, custom setup.py hooks, or even a custom backend—run into the same root issue: the code that needs the original project root no longer runs where those paths make sense, and the path to that root is not available by default.

Why this happens

PEP 517 build frontends move or copy the source tree into a backend-controlled working directory to build distributions in a clean environment. That design means paths that are only valid relative to the original working directory will break unless you explicitly communicate the original root to the backend. Passing absolute paths is undesirable, and installing your C++ artifacts into global prefixes (like /usr) is out of scope for a per-project build. So the core problem is getting the original project root into the build backend, then rewriting the extension’s include_dirs and library_dirs to absolute paths anchored at that root.

A pragmatic fix without rearranging the repo

The approach hinges on two pieces. First, use the build frontend’s configuration channel to pass the original working directory into the backend. Second, implement a thin wrapper over setuptools.build_meta that reads this value and amends the extension configuration before compilation. This keeps the repository structure intact and avoids introducing new external tooling.

Start by invoking the build with a configuration argument that records the current directory. A Makefile can make this one-liner friendly, but the essence is:

python -m build -C project-root=`pwd`

Next, point your project at a tiny custom backend, and ensure setuptools uses a modified build_ext that adjusts relative include/library paths. The updated Python package layout looks like this:

python
├── Makefile
├── pyproject.toml
├── README.md
├── my-python-package
│   ├── __init__.py
│   ├── _build_hooks.py
│   ├── py.typed
│   └── my-python-module.pyi
└── src
    └── my-python-module.cc

In pyproject.toml, configure the backend and the custom build_ext class. Relative paths for include-dirs and library-dirs can remain in your ext-modules definition; the backend will make them absolute at build time.

[build-system]
requires = ["setuptools >=80.9"]
build-backend = "_build_hooks"
backend-path = ["my-python-package"]
[tool.setuptools]
packages = ["my-python-package"]
[tool.setuptools.cmdclass]
build_ext = "_build_hooks.build_ext"

The custom backend is a light layer over setuptools.build_meta. It reads project-root from the frontend’s -C option, then amends include_dirs and library_dirs so relative entries become absolute by joining them with the recorded root. The primary logic lives in build_wheel and an overridden initialize_options of build_ext.

from setuptools.build_meta import *
from setuptools.build_meta import build_wheel as _build_wheel_impl
from setuptools.command.build_ext import build_ext as _build_ext_base
import typing
_workdir_hint: str = ""
def _abspath_or_join(p: str) -> str:
    if p.startswith("/"):
        return p
    return f"{_workdir_hint}/{p}"
def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
    global _workdir_hint
    if config_settings:
        _workdir_hint = config_settings.get("project-root", "")
    return _build_wheel_impl(wheel_directory, config_settings, metadata_directory)
class _BuildExtAmend(_build_ext_base):
    @typing.override
    def initialize_options(self):
        super().initialize_options()
        for ext in self.distribution.ext_modules:
            ext.include_dirs = list(map(_abspath_or_join, ext.include_dirs))
            ext.library_dirs = list(map(_abspath_or_join, ext.library_dirs))
# expose the symbol referenced by tool.setuptools.cmdclass
build_ext = _BuildExtAmend

What this achieves

When python -m build runs, the frontend passes project-root into the backend via config_settings. The overridden build_wheel stores that directory, and the custom build_ext makes include_dirs and library_dirs absolute before compilation. Because the extension objects are adjusted inside the backend, the separate build location no longer breaks path resolution, and you avoid hardcoding absolute paths or installing system-wide dependencies. The rest of your extension sources can continue to be copied as usual.

Notes on alternatives

PEP 517 suggests a self-contained source tree, which could be interpreted as including the C++ library sources directly in the Python package. That would eliminate cross-tree paths entirely. Another route would be adopting a backend that integrates with CMake, such as scikit-build, which can streamline mixed C++/Python builds. In the scenario considered here, the goal was to avoid restructuring the repo or switching build systems, so the minimal backend wrapper provided the necessary control with minimal churn.

Why this matters

Build reproducibility and clarity are critical when distributing binary wheels. If the extension build silently depends on the caller’s current working directory or assumes in-tree artifacts are present in the backend’s temporary workspace, builds become flaky and opaque. Making the project root an explicit input and normalizing paths inside the backend keeps the build predictable, particularly when moving between developer machines and CI or when toggling build frontend options like isolation.

Wrap-up

If your CPython extension relies on a sibling C++ library built outside the backend’s working directory, solve the path problem at the source: pass the project root through the build frontend and fix include and library paths in a small, focused backend layer. Keep relative paths in pyproject.toml, avoid absolute system paths, and wire up a custom build_ext to normalize them. The build command stays ergonomic with a Makefile wrapper, the repository layout remains unchanged, and setuptools gains just enough context to compile the extension reliably.