You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.rst
+8-7Lines changed: 8 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,15 +7,16 @@ Changelog
7
7
8
8
* Support Python 3.15.
9
9
10
-
* Shuffle tests before other plugins’ ``pytest_collection_modifyitems`` hooks run.
10
+
* Shuffle tests in a wrapper around the ``pytest_collection_modifyitems`` hook, guaranteeing that the shuffle runs before all other plugins’ implementations of the hook.
11
11
12
-
Previously, pytest-randomly used a plain ``tryfirst`` hook, so another plugin using ``tryfirst``, such as pytest-django, could reorder tests before or after pytest-randomly’s shuffle, depending on the plugin registration order.
13
-
Since that order can vary between environments, this would make the final test order non-repeatable.
12
+
Previously, pytest-randomly shuffled in a plain ``tryfirst`` hook implementation.
13
+
When another plugin also implemented the hook with ``tryfirst``, as pytest-django does, whichever plugin pytest happened to register later ran first.
14
+
Registration order comes from package metadata on disk, which can differ between seemingly identical environments — even two containers built from the same ``Dockerfile``, or the same virtual environment after reinstalling a package.
15
+
As a result, the same seed could yield different test orders in different environments.
16
+
Worse, in environments where pytest-randomly ended up shuffling last, it silently destroyed the other plugin’s ordering — for pytest-django, the grouping of database tests that mirrors Django’s test runner (non-transactional database tests, then transactional ones, then the rest).
14
17
15
-
Now, pytest-randomly marks its hook as a hook wrapper, making it run before any other plugin’s ``pytest_collection_modifyitems`` hooks, regardless of registration order.
16
-
Plugins that group tests with a stable sort can apply their grouping on top of the shuffled order, making the final test order reproducible from the seed.
17
-
18
-
This is a low-impact breaking change because it you may not be able to reprduce a test order from the last pytest-randomly version by reusing a ``--randomly-seed`` value.
18
+
Now the shuffle always runs first, and plugins that group tests with a stable sort apply their grouping on top of the shuffled order, so the final order is reproducible from the seed alone.
19
+
In environments that previously hit the reversed hook order, upgrading changes the test order for a given seed — restoring both reproducibility and other plugins’ grouping.
0 commit comments