Pytest vs tempfile.gettempdir: Why TMP changes don't stick and how to set the temp directory correctly

Under pytest, tempfile.gettempdir caches early, so changing TMP won't work. Learn why TMPDIR/TEMP take precedence and how to set the temp dir before tests.

When code that tweaks TMP at runtime works from the command line but fails under pytest, it feels like something in the test runner is “messing” with the environment. The reality is simpler and more subtle: tempfile caches the temp directory on first use, and tests usually trigger that earlier than your test function does.

Minimal reproduction

The following snippet expects that setting TMP will immediately change what tempfile.gettempdir() returns:

import os
import tempfile
def check_tmp_under_test():
    os.environ["TMP"] = "/home/fedora"
    assert tempfile.gettempdir() == "/home/fedora"
    print(tempfile.gettempdir())
check_tmp_under_test()

Run it directly and you’ll see /home/fedora. Run it via pytest, and it may print /tmp or error on the assertion. Same machine, same Python, different outcome.

What is actually happening

The key detail is process-wide caching inside the tempfile module. The first call in the process to tempfile.gettempdir() determines the directory, and that value is cached for the lifetime of the process. Changing os.environ after that first call does not affect the cached result.

After the first call to tempfile.gettempdir() the value it reads is cached — meaning that if you change os.environ after the first call in the process, the value returned first is still returned.

You can observe the cache in an interactive session:

import tempfile, os
print(tempfile.gettempdir())  # e.g. '/tmp'
os.environ["TMP"] = "/tmp/xis"
print(tempfile.gettempdir())  # still '/tmp' because it was cached

Under pytest, a call to tempfile.gettempdir() often happens before your test function runs. It can be triggered by the test runner itself, a plugin, or any code that creates a temp file or directory, because those code paths use the same cached value. Once cached, later changes to TMP inside the test won’t take effect.

There is another piece to keep in mind. The tempfile module consults multiple environment variables, and some have higher priority than others. TMPDIR and TEMP take precedence over TMP. If either TMPDIR or TEMP is set in the test process, changing TMP won’t matter.

Internally, there are private helpers such as tempfile._candidate_tempdir_list() that reflect the current environment on each call, but both tempfile.gettempdir() and tempfile._gettempdir() stop being dynamic after the first call in the process. There is no supported way to reset that cache mid-process.

Solution

Make the environment deterministic before the test process starts. Set the desired temp directory in the OS environment when launching pytest, rather than inside the test function. That ensures the first call to tempfile in the process sees the right value, and the cached result matches your expectation. Also ensure that higher-precedence variables like TMPDIR or TEMP are not set to conflicting values.

export TMP=/home/fedora
uv run pytest pytest.py

With TMP set at process start, tempfile.gettempdir() will cache /home/fedora on its first use, and your assertions will align in both direct runs and under pytest.

Why this matters

Tests live in long-running processes that do more than your code does. Runners, plugins, and utilities may touch tempfile early, and because of caching, that early touch fixes the temp directory for everything that follows. Relying on mutating os.environ mid-test for temp paths is brittle and leads to non-deterministic failures.

Takeaways

Set TMP before the Python process starts if you need a specific temp directory. Remember that TMPDIR and TEMP override TMP, so verify they aren’t set to something else in the test environment. Don’t count on changing os.environ["TMP"] during a test to update tempfile.gettempdir(); once cached, the value is fixed for the lifetime of the process. If you need to inspect what tempfile would consider at this moment without committing to the cache, you can look at the dynamic candidate list via private utilities, but there’s no supported reset for the cached value, so controlling the environment at process start is the reliable path.