Make it possible to run Pytest in parallel#1291
Draft
mhucka wants to merge 10 commits intoquantumlib:mainfrom
Draft
Make it possible to run Pytest in parallel#1291mhucka wants to merge 10 commits intoquantumlib:mainfrom
mhucka wants to merge 10 commits intoquantumlib:mainfrom
Conversation
Setting the seed in conftest.py like this makes it possible to run pytest in parallel with the `-n` option from pytest-xdist.
Some of the test code did not work with parallel pytest execution because it attempted to write to and delete the same files in the shared `src/openfermion/testing/data` directory. Specifically, `SaveLoadOperatorTest` and `MolecularDataTest` used fixed filenames:(like `test_file.data` and `dummy_molecule.hdf5`) which led to collisions and errors during parallel execution. The changes implemented here are: * Refactor `SaveLoadOperatorTest` in `operator_utils_test.py` to use a unique temporary directory for each test instance through the use of `tempfile.mkdtemp()`. * Refactor `MolecularDataTest` in `molecular_data_test.py` to use a temporary directory for tests that perform disk writes (`test_dummy_save` and `test_abstract_molecule`).
Although `check/pytest` accepts command line arguments, `pytest-and-incremental-coverage` does not, so we can't instruct contributors to add `-n auto` to it to run it in parallel. Since it's something that is less commonly run by contributors, I took the more expedient route of simply adding the parallelism flag directly to the script.
Now that parallel Pytest execution works for OpenFermion, we can tell users about it.
Turns out the use of the `pytest-randomly` plugin and using `np.random.default_rng` seems to make it possible to run pytest in parallel. Evidently, that seeds every test individually based on a master seed, and this solves the xdist collection issue automatically without creating inter-test dependencies.
Regenerate dependency files after adding pytest-randomly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes 2 sets of changes that together allow parallel Pytest execution via the
pytest-xdistplugin.Set a fixed random seed for testing, in
src/openfermion/conftest.py. This is necessary forpytest-xdistworker processes to be able to perform identical test collections. Otherwise, different workers might generate different values and you end up with "Inconsistent collection" errors during execution.Change some tests that were writing to fixed-path files and folders. Those needed to be changed to use temporary directories so that parallel Pytest processes don't collide trying to write to the same things.