Pytest Fixture Pattern to Precompute, Pickle, and Reuse Expensive Artifacts Across Test Runs

Use a pytest fixture to precompute, pickle, and persist expensive results at the project root—automate first-run setup and speed up deterministic test runs.

Precomputing an expensive result and reusing it across test runs is a pragmatic way to keep feedback loops tight. The catch: when you rely on a manual first-run initializer, test automation becomes fragile. With pytest, you don’t need to toggle code on and off; you can let a fixture generate and persist the data the first time, then reuse it on subsequent runs.

What goes wrong with the manual approach

The workflow looks straightforward at first: run the expensive solver once, pickle the result, then load it in tests. But this breaks down when you have to remember to “prime” the pickle manually, or when relative paths cause duplicate files depending on where pytest was invoked. You want a first-run setup that happens automatically and persists locally across sessions.

Minimal example that illustrates the pain

Here’s a compact example of the pattern where the heavy computation must be executed once and reused afterward.

import time
import pickle
class Answer:
    """Represents a solution produced by a time-consuming routine."""
def crunch_hard_task() -> Answer:
    time.sleep(1)
    return Answer()
def snapshot_result() -> None:
    """Run the solver once and persist the outcome for feature development."""
    outcome = crunch_hard_task()
    with open('solved.pickle', 'wb') as fh:
        pickle.dump(outcome, fh)
def feature_under_test(solved: Answer) -> None:
    """Use the prepared solution."""
    print(solved)
def test_feature() -> None:
    # snapshot_result()  # This must be manually toggled on first run.
    with open('solved.pickle', 'rb') as fh:
        restored = pickle.load(fh)
    assert feature_under_test(restored) is None

Root cause

The tests assume the file already exists. If it doesn’t, they fail or force you to toggle an initializer. Additionally, placing the pickle at a relative path means invoking pytest from different subdirectories can regenerate the file in multiple locations. The workflow lacks a reliable, automated first-run path and a canonical location for the persisted data.

Automating the first run with a pytest fixture

Pytest fixtures can run arbitrary setup logic. That’s exactly what we need: check if the pickle exists, load it if available, otherwise compute once, save, and return it. To avoid duplicate files when running tests from nested folders, reference the project root directory via pytest’s configuration object.

import pickle
from pathlib import Path
import pytest
class Answer:
    """Represents a solution produced by a time-consuming routine."""
def crunch_hard_task() -> Answer:
    # Simulate a costly computation
    import time
    time.sleep(1)
    return Answer()
@pytest.fixture
def solved_artifact(pytestconfig) -> Answer:
    # Create a single canonical location at the project root
    pickle_file = pytestconfig.rootpath / "solved.pickle"
    if pickle_file.exists():
        with pickle_file.open("rb") as fh:
            return pickle.load(fh)
    print("Solving problem, please be patient")
    computed = crunch_hard_task()
    with pickle_file.open("wb") as fh:
        pickle.dump(computed, fh)
    return computed
def feature_under_test(solved: Answer) -> None:
    print(solved)
def test_feature(solved_artifact: Answer) -> None:
    assert feature_under_test(solved_artifact) is None

This removes the manual toggle. On the first run, the fixture generates and caches the pickle in the project’s base directory; on subsequent runs, it just loads the file. Because the path is rooted at the project root, it won’t multiply files when running tests from subdirectories.

Optional variants

If you want the computation to occur only once per pytest invocation, regardless of how many tests use it, consider a session-scoped fixture. It still runs every time you launch pytest, but only once per session.

@pytest.fixture(scope="session")
def solved_artifact(pytestconfig) -> Answer:
    pickle_file = pytestconfig.rootpath / "solved.pickle"
    if pickle_file.exists():
        with pickle_file.open("rb") as fh:
            return pickle.load(fh)
    print("Solving problem, please be patient")
    computed = crunch_hard_task()
    with pickle_file.open("wb") as fh:
        pickle.dump(computed, fh)
    return computed

Depending on how stable your data model is, another practical approach is to keep the pickle in source control and regenerate it out-of-band when required. Whether that’s a good fit depends on how rarely the underlying structure changes.

There is also a built-in cache system in pytest that can be a direct match for persisting data across runs without manually maintaining a file. See the documentation for the config cache object: https://docs.pytest.org/en/stable/how-to/cache.html#the-new-config-cache-object

Why this matters

Automating first-run setup makes test runs deterministic and repeatable. It removes a human-in-the-loop step that’s easy to forget and keeps artifacts in a single, predictable place. That’s especially valuable when tests are spread across subpackages and executed from different working directories; using the project root for persisted state avoids fragmentation.

Takeaways

Use a pytest fixture that checks for a persisted artifact and generates it on demand. Anchor the storage path to the project root via pytest configuration so the file is shared across all test invocations, regardless of the current working directory. If you need once-per-session behavior, scope the fixture to the session; if you want to avoid managing files at all, evaluate pytest’s cache system. And if your solution object is extremely stable, keeping the pickle under version control can be a simple alternative. The result is zero manual toggles, faster iteration, and a predictable, durable setup that survives across sessions.