How to Reuse a Single httpx.Client for Connection Pooling: Correct Context Management and Lifetime

Understand httpx connection pooling with httpx.Client: open once, reuse safely, avoid RuntimeError from multiple context entries, and manage client lifetime.

Working with httpx often raises a practical question: how exactly does connection pooling relate to httpx.Client and its lifetime? The docs recommend using a client for pooling, but the nuances of opening, closing, and context management can be confusing when you want to share a single client across different places in your code.

Reproducing the pitfall

A common intuition is to keep a single shared client and just enter it via a context manager whenever needed. That approach looks convenient, but it runs into a hard restriction.

import httpx
pooled_agent = httpx.Client()
with pooled_agent as c1:
    # do something
    pass
with pooled_agent as c2:
    # do something else
    pass

This pattern tries to open the same client instance more than once. That does not work and results in a runtime error when you hit the second with-block.

RuntimeError: Cannot open a client instance more than once.

What actually happens

The examples above show that entering a context on the same httpx.Client multiple times is invalid. A client cannot be reopened. This directly affects how you approach connection pooling: if you want to reuse connections, you need a single opened client that remains open for the duration of your work and is shared where needed, rather than repeatedly opening and closing it.

The working pattern: open once, reuse

The safe and effective pattern is to create and open a client once, then pass that instance around while it stays open. The following example demonstrates this with a thread pool: one client is opened, reused across workers, and closed only after all work is done.

import concurrent.futures
import httpx
ENDPOINTS = [f"http://example.com/{i}" for i in range(20)]
def fetch_page(sess: httpx.Client, link: str):
    resp = sess.get(link)
    return resp.text
with httpx.Client() as sess:
    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as pool:
        pending = {pool.submit(fetch_page, sess=sess, link=addr): addr for addr in ENDPOINTS}
        for done in concurrent.futures.as_completed(pending):
            done.result()
            print(f"{pending[done]} done")

Now compare that with the tempting but incorrect approach that attempts to open the same client instance in multiple with-blocks, even concurrently. That leads straight to the error shown earlier.

import httpx
reused_client = httpx.Client()
with reused_client as first:
    with reused_client as second:
        pass
# Raises:
# RuntimeError: Cannot open a client instance more than once.

Allowing callers to supply a Client

Sometimes you want a class to use httpx.Client internally while allowing advanced users to provide their own client instance, reuse it across calls, and manage its lifetime themselves. One way to do that is to forward attribute access to the provided client and avoid closing it on context exit. The wrapper below emits a warning if the provided client remains open after exit, leaving ownership of closing to the caller.

import warnings
import httpx
warnings.simplefilter("always")
class _ClientFacade:
    def __init__(self, engine: httpx.Client) -> None:
        self.engine = engine
    def __getattr__(self, name):
        return getattr(self.engine, name)
    def __enter__(self):
        return self.engine.__enter__()
    def __exit__(self, exc_type, exc_value, tb):
        if not self.engine.is_closed:
            warnings.warn(f"httpx.Client instance '{self.engine}' is still open. ")
class Fetcher:
    def __init__(self, engine: httpx.Client | None = None) -> None:
        self.engine = httpx.Client() if engine is None else _ClientFacade(engine)
    def fetch(self, target: str) -> httpx.Response:
        with self.engine:
            reply = self.engine.get(target)
            reply.raise_for_status()
        return reply
external = httpx.Client()
svc = Fetcher(engine=external)
out = svc.fetch("https://www.example.com")
print(out)  # 200
print(svc.engine.is_closed)  # False
external.close()
print(svc.engine.is_closed)  # True

This approach lets you respect ownership: when a client is supplied from the outside, it’s reused and not implicitly closed by your class. When no client is supplied, a new one is created and managed internally.

Why this matters

Mismanaging the client lifecycle can cause immediate runtime errors and defeats the point of reusing a shared httpx.Client. Keeping a single opened client and sharing it across the places that need it prevents those errors and aligns with the idea of maintaining a reusable pool during the lifetime of your work.

Takeaways

If you want connection reuse with httpx, keep the lifecycle of your client simple. Open one httpx.Client, reuse it while it’s open, and close it when you are done. Do not attempt to enter the same client via multiple context managers at different times or in parallel. When exposing functionality in a class, allow passing an existing client so that advanced users can control ownership and closing explicitly.