pytest-leak-finder bisects test suites to find state-leaking tests

pytest-leak-finder is a pytest plugin built to answer a familiar problem: a test passes when run alone but fails when the full suite runs, because some earlier test leaves behind state it never cleans up. The plugin's own description frames the puzzle plainly: some previous test keeps things dirty, but it is not obvious which one, and there could be more than one. Rather than making a developer bisect the suite by hand, the plugin automates it with a binary search over the tests that ran before the failing one, the target, in the same way git bisect narrows down a bad commit. On the first run, pytest-leak-finder marks the failing test as the target and stops the session so the search can begin. Each following run splits the remaining candidate tests in half: if the target still fails with that half included, the leak is in that half and the search goes deeper there; if the target passes, the tool discards that half and tries the other one, recording the path taken as a short string of steps such as "a" or "ba". The README walks through a worked example with a demo suite of 6 tests where test5 fails only in the full run. Bisecting that suite takes three pytest-leak-finder runs: the first sets test5 as the target and stops the session; the second, step "a" (test1 and test2), leaves the target passing, so the tool discards that half and moves to step "ba"; the third runs test3 alongside the target, reproduces the failure, and the tool reports test3 as the culprit.

Key facts

pytest-leak-finder is a pytest plugin that identifies which earlier test in a suite leaves behind state that makes a later test fail only when the full suite runs.
It works by binary search, in the style of git bisect, over the tests that ran before the failing target test.
On the first run it sets the failing test as the target and stops the session to begin the search.
In the worked example, a 6 test demo suite, it narrows the candidates across three runs (steps "a" then "ba") and identifies test3 as the leaking test.
The plugin's own framing of the problem is a working hypothesis, not a guarantee: some previous test keeps things dirty, possibly more than one.

Why it matters

Tests that pass alone but fail inside a full suite run are a common and time consuming problem in Python projects, usually tracked down by manually re-running subsets of tests. pytest-leak-finder turns that manual trial and error into an automated bisection, the same idea behind git bisect applied to test order dependence instead of commit history.

Who it affects

Python developers who use pytest and run into flaky, order dependent suites, specifically the case where a test is green in isolation (or under --lf) but red when the whole suite runs, a sign that some other test is leaking state into it.

How to use it

The plugin adds a --leak-finder flag to pytest. The first invocation on a suite with a known failure sets that failing test as the target and stops. Re-running the same command repeatedly narrows the set of preceding tests included in the run, following the bisection path, until the tool reports which specific test is the leak, as shown in the worked example that resolves in three runs on a 6 test suite.

How solid is it

The only material available is the plugin's own description and a single walked-through demo suite of 6 tests; there is no information on the author or maintainer's identity, license, installation steps, supported pytest versions, or real-world adoption. The Hacker News submission carries 17 points and no comments, so there is no outside discussion to corroborate how it performs on larger or more complex suites.

Risks and caveats

The bisection approach is built around finding a single leaking test; the author's own framing allows for the possibility that more than one previous test contributes to the dirty state, a case the walked-through example does not cover. The source gives no detail on how the tool behaves with non-deterministic leaks, large suites, or multiple simultaneous culprits, and no performance or false-positive data is provided.

“My two cents that some previous test keeps the things dirty. But wich one/s, maybe the previous are a lot, right?”

— pytest-leak-finder's project description