chore: clean cut experiments - #524
Conversation
ae7e408 to
739702e
Compare
Vulnerability Scan ResultsDetails |
Consolidated Tests Results 2026-04-27 - 09:46:18Test ResultsDetails
test-reporter: Run #1675
🎉 All tests passed!TestsView All Tests
🍂 No flaky tests in this run. Github Test Reporter by CTRF 💚 🔄 This comment has been updated |
81de7ab to
3222620
Compare
There was a problem hiding this comment.
Pull request overview
This PR restructures the workspace by removing the former core/threshold (threshold-fhe) crate and consolidating experiment/benchmark binaries and related support code into a new core/threshold-experiments crate, while updating documentation, build tooling, and dependencies accordingly.
Changes:
- Replace
threshold-fhewiththreshold-experimentsin the workspace and update references across docs, CI, Docker, and scripts. - Move/gather experiment, KAT, and benchmark utilities under
core/threshold-experiments, and refactor choreography helpers into shared modules. - Relocate TLS certificate generation helpers into
threshold-networkingand adjust core/service + client usage.
Reviewed changes
Copilot reviewed 84 out of 131 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/generate-test-material/Cargo.toml | Swap dependency to threshold-execution for feature unification |
| docs/guides/threshold-benchmark.md | Update paths to core/threshold-experiments |
| docs/guides/kms-server-bin.md | Update kms-gen-keys feature flags |
| docs/explanations/node.md | Update Threshold Service README link |
| docs/explanations/network_doc.md | Update insecure-mode wording/paths |
| docker/core/service/Dockerfile | Remove threshold-fhe/testing build flag |
| core/threshold/src/lib.rs | Remove old threshold-fhe lib entrypoint |
| core/threshold/src/grpc/mod.rs | Remove old grpc module stub |
| core/threshold/Cargo.toml | Remove old threshold-fhe manifest |
| core/threshold-types/Cargo.toml | Drop unused deps (aes-prng, rand) |
| core/threshold-networking/src/tls.rs | Adjust CN/issuer validation to support core certificates |
| core/threshold-networking/src/lib.rs | Export tls_certs module |
| core/threshold-networking/Cargo.toml | Add deps needed for tls_certs (clap/rcgen/tokio fs) |
| core/threshold-experiments/tests/integration_redis.rs | Serialize tests; remove cfg gating on imports/tests |
| core/threshold-experiments/test_scripts/tfhe_test_small_session.sh | Remove explicit choreographer feature from build |
| core/threshold-experiments/test_scripts/tfhe_test_large_session.sh | Remove explicit choreographer feature from build |
| core/threshold-experiments/test_scripts/tfhe_reproducible_small_session.sh | Remove explicit choreographer feature from build |
| core/threshold-experiments/test_scripts/bgv_test_script_real_dkg.sh | Remove explicit choreographer feature from build |
| core/threshold-experiments/test_scripts/bgv_test_script_fake_dkg.sh | Remove explicit choreographer feature from build |
| core/threshold-experiments/src/zk_utils.rs | Add shared ZK-PoK utilities for KATs/benches |
| core/threshold-experiments/src/utils.rs | Add SHA-256 hash checking helper |
| core/threshold-experiments/src/tests/config/error_conf.toml | Add intentionally-invalid config fixture |
| core/threshold-experiments/src/tests/config/ddec_test.toml | Add config fixture for PartyConf tests |
| core/threshold-experiments/src/tests/config/ddec_no_peers.toml | Add config fixture without peers |
| core/threshold-experiments/src/lib.rs | New crate lib module exports |
| core/threshold-experiments/src/conf/party.rs | Party configuration + TLS helper code |
| core/threshold-experiments/src/conf/mod.rs | Make choreo module unconditional |
| core/threshold-experiments/src/conf/choreo.rs | Define NetworkTopology locally |
| core/threshold-experiments/src/choreography/tfhe_rs/strategies.rs | Refactor routing helper + secure choreography alias |
| core/threshold-experiments/src/choreography/tfhe_rs/requests.rs | Define TFHE choreography request DTOs |
| core/threshold-experiments/src/choreography/tfhe_rs/mod.rs | TFHE choreography module layout |
| core/threshold-experiments/src/choreography/tfhe_rs/grpc.rs | Refactor grpc service; move shared helpers to server_utils |
| core/threshold-experiments/src/choreography/tfhe_rs/choreographer.rs | Refactor client runtime usage into shared utility |
| core/threshold-experiments/src/choreography/server_utils.rs | New shared network/memory stats + session helpers |
| core/threshold-experiments/src/choreography/server.rs | Server runner refactor + remove duplicated alias |
| core/threshold-experiments/src/choreography/mod.rs | New choreography top-level module map |
| core/threshold-experiments/src/choreography/client_utils.rs | Shared client runtime + status check loop |
| core/threshold-experiments/src/choreography/bgv/strategies.rs | BGV routing helper for experimental choreography |
| core/threshold-experiments/src/choreography/bgv/requests.rs | Update BGV request types to use threshold-experimental |
| core/threshold-experiments/src/choreography/bgv/mod.rs | Add BGV strategies module |
| core/threshold-experiments/src/choreography/bgv/grpc.rs | Port BGV choreography service off threshold-fhe |
| core/threshold-experiments/src/choreography/bgv/choreographer.rs | Use shared ChoreoRuntime + new imports |
| core/threshold-experiments/src/bin/threshold-tfhe/mobygo.rs | Update imports to threshold-experiments choreography layout |
| core/threshold-experiments/src/bin/threshold-tfhe/moby.rs | Update server runner and allocator path |
| core/threshold-experiments/src/bin/threshold-gen-tls-certs.rs | Use threshold_networking::tls_certs |
| core/threshold-experiments/src/bin/threshold-bgv/stairwayctl.rs | Use threshold-experiments client utils + requests |
| core/threshold-experiments/src/bin/threshold-bgv/stairway.rs | Add BGV MPC party binary |
| core/threshold-experiments/src/bin/non-threshold/zk-pok-kat.rs | Update imports to new shared zk/hash utils |
| core/threshold-experiments/src/bin/non-threshold/tfhe-kat.rs | Update hash util import |
| core/threshold-experiments/src/bin/benches/gen-experiment.rs | Add templated experiment config generator |
| core/threshold-experiments/src/allocator.rs | Add shared peak allocator holder |
| core/threshold-experiments/experiments/templates/otel-collector-config.yaml | Add otel collector template |
| core/threshold-experiments/experiments/templates/docker-compose.yml.j2 | Update hostnames/cert paths for core cert model |
| core/threshold-experiments/experiments/templates/conf.toml.j2 | Update logical addresses to core1.* |
| core/threshold-experiments/examples/distributed_decryption.rs | Update crate name in example docs |
| core/threshold-experiments/docker/scripts/wan.sh | Add netem WAN script |
| core/threshold-experiments/docker/scripts/runinallcontainers.sh | Helper to exec across containers |
| core/threshold-experiments/docker/scripts/off.sh | Remove qdisc helper |
| core/threshold-experiments/docker/scripts/lan10.sh | Add LAN 10G netem script |
| core/threshold-experiments/docker/scripts/lan1.sh | Add LAN 1G netem script |
| core/threshold-experiments/docker/nist_testing.dockerfile | Install bins from core/threshold-experiments |
| core/threshold-experiments/docker/local.dockerfile | Install bins from core/threshold-experiments |
| core/threshold-experiments/coverage.sh | Update crate name parsing for coverage artifacts |
| core/threshold-experiments/config/mobygo.toml | Add example choreographer config |
| core/threshold-experiments/config/moby.toml | Add example moby party config |
| core/threshold-experiments/benches/non-threshold/utilities.rs | Update allocator import path |
| core/threshold-experiments/benches/non-threshold/tfhe-zk-pok/speed.rs | Update zk_utils import path |
| core/threshold-experiments/benches/non-threshold/tfhe-zk-pok/memory.rs | Update zk_utils + allocator import path |
| core/threshold-experiments/benches/non-threshold/tfhe-rs/speed/keygen.rs | Add TFHE keygen bench |
| core/threshold-experiments/benches/non-threshold/tfhe-rs/speed/erc20.rs | Add TFHE ERC20-style bench |
| core/threshold-experiments/benches/non-threshold/tfhe-rs/speed/basic_ops.rs | Add TFHE basic ops bench |
| core/threshold-experiments/benches/non-threshold/tfhe-rs/memory/keygen.rs | Add TFHE keygen memory bench |
| core/threshold-experiments/benches/non-threshold/tfhe-rs/memory/erc20.rs | Add TFHE ERC20 memory bench |
| core/threshold-experiments/benches/non-threshold/tfhe-rs/memory/basic_ops.rs | Add TFHE basic ops memory bench |
| core/threshold-experiments/benches/non-threshold/bgv/speed/keygen.rs | Add BGV keygen speed bench |
| core/threshold-experiments/benches/non-threshold/bgv/speed/basic_ops.rs | Add BGV basic ops speed bench |
| core/threshold-experiments/benches/non-threshold/bgv/memory/keygen.rs | Add BGV keygen memory bench |
| core/threshold-experiments/benches/non-threshold/bgv/memory/basic_ops.rs | Add BGV basic ops memory bench |
| core/threshold-experiments/benches/decoding.rs | Add decoding/error-correction benchmarks |
| core/threshold-experiments/benches/ddec.rs | Add distributed decryption benches |
| core/threshold-experiments/benches/ceremony.rs | Add CRS ceremony bench |
| core/threshold-experiments/benches/bit_dec.rs | Add bit decomposition benches |
| core/threshold-experiments/benches/bgv.rs | Add BGV benches and conversions bench |
| core/threshold-experiments/benches/bench_switch_and_squash.rs | Add switch-and-squash bench |
| core/threshold-experiments/benches/bench_prss.rs | Add PRSS bench |
| core/threshold-experiments/benches/algebra.rs | Add interpolation/Lagrange benches |
| core/threshold-experiments/README.md | Add minimal crate README |
| core/threshold-experiments/NIST_scripts/non-threshold-zk-pok-bench.sh | Update run-from path |
| core/threshold-experiments/NIST_scripts/non-threshold-tfhe-bench.sh | Update benches path reference |
| core/threshold-experiments/NIST_scripts/non-threshold-kat.sh | Update TFHE KAT binary name |
| core/threshold-experiments/NIST_scripts/build.sh | Update install path to new crate |
| core/threshold-experiments/NIST_scripts/.gitignore | Ignore bench/KAT outputs |
| core/threshold-experiments/Makefile.toml | Update docker build features/flags + cert generation args |
| core/threshold-experiments/Cargo.toml | Add new crate manifest |
| core/threshold-experiments/.gitignore | Add crate-local ignores |
| core/threshold-experiments/.dockerignore | Add crate-local docker ignores |
| core/threshold-experiments/.cargo/config.toml | Force git fetch with CLI |
| core/threshold-experimental/src/lib.rs | Remove choreographer module exports |
| core/threshold-experimental/src/constants.rs | Export INPUT_PARTY_ID publicly |
| core/threshold-experimental/src/bgv/utils.rs | Make gen_key_set public |
| core/threshold-experimental/src/bgv/ddec.rs | Make noise_flood_decryption public |
| core/threshold-experimental/benches/non-threshold/utilities.rs | Remove old bench utilities file |
| core/threshold-experimental/Cargo.toml | Drop bins/benches previously tied to choreographer features |
| core/threshold-execution/src/online/triple.rs | Gate multi-degree tests behind slow_tests |
| core/threshold-execution/src/online/preprocessing/orchestration/producers/triples_producer.rs | Gate multi-degree tests behind slow_tests |
| core/threshold-execution/src/online/preprocessing/orchestration/producers/randoms_producer.rs | Gate multi-degree tests behind slow_tests |
| core/threshold-execution/src/online/preprocessing/orchestration/producers/bits_producer.rs | Gate multi-degree tests behind slow_tests |
| core/threshold-execution/src/online/gen_bits.rs | Gate multi-degree tests behind slow_tests |
| core/threshold-execution/src/malicious_execution/mod.rs | Remove degree-specific cfg on module |
| core/threshold-execution/src/large_execution/single_sharing.rs | Gate expensive tests behind slow_tests |
| core/threshold-execution/src/endpoints/reshare_sk.rs | Gate multi-degree tests behind slow_tests |
| core/threshold-execution/src/endpoints/keygen.rs | Gate multi-degree tests behind slow_tests |
| core/threshold-execution/src/endpoints/decryption_non_wasm.rs | Gate multi-degree tests behind slow_tests |
| core/threshold-execution/Cargo.toml | Simplify features; enable all degrees via dev-dep |
| core/service/tests/integration_test.rs | Remove cert path integration test tied to removed crate |
| core/service/src/util/key_setup/mod.rs | Use threshold_networking::tls_certs |
| core/service/src/engine/threshold/service/kms_impl.rs | Update log wording to new path/name |
| core/service/src/client/test_tools.rs | Update comments to new path/name |
| core/service/src/client/js_api.rs | Update wasm note to new path/name |
| core/service/src/bin/kms-gen-tls-certs.rs | Use threshold_networking::tls_certs |
| core/service/Cargo.toml | Remove threshold-fhe dep; tweak features and wasm deps |
| core-client/src/mpc_context.rs | Use threshold_networking::tls_certs |
| core-client/Cargo.toml | Adjust deps/features; drop threshold-fhe |
| bc2wrap/src/lib.rs | Update comment referencing new constant location |
| Cargo.toml | Replace member + workspace deps (threshold-experiments, rename threshold-experimental key) |
| Cargo.lock | Lockfile updates following crate move/dep changes |
| .github/workflows/main.yml | Point threshold job to new crate path/name and args |
| .github/workflows/README.md | Update CI docs to new path |
Comments suppressed due to low confidence (1)
core/threshold-experiments/NIST_scripts/build.sh:62
- The
TARGET_DIRassignment is missing a closing quote (and currently includes a newline), which will make the script fail to run. Fix the quoting soTARGET_DIRis a valid string path (and consider normalizing the trailing slash).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
8d16ae1 to
2d1c1bf
Compare
dd23
left a comment
There was a problem hiding this comment.
LGTM, modulo the small open points.
144dccf to
a70a02d
Compare
* chore(ci): parallelize tests, use bigger runners, CoW filesystem Consolidates the CI reform work from dvdplm/chore/light-touch-ci-reform-3: - Run jobs with as much parallelism as possible (10-way for the slowest) - Use 64-core runners for the heaviest test groups - BTRFS CoW-enabled fs for faster test-material copies - Per-job test_threshold_slow split out for parallel execution - Build binaries in --profile test to share artifacts with the test step - Remove grcov, bump nextest to 0.9.133 - Skip k8s tests on vanilla PRs - Generate test material for wasm tests - Path filters and crate names updated for the post-#524 rename (core/threshold/ -> core/experiments/, threshold-experimental -> threshold-bgv) * chore: add a helper to ThresholdTestEnv to destructure more ergonomically * chore: add into_parts/create)internalk_client helpers to CentralizedTestEnv * chore: intra-test cleanup fix to include compressed xof keys and decompression key * chore: only re-generate material for Testing, leave Default to trust fixture * chore: un-serialize centralzied core/service/ tests * chore: root out #[serial] from threshold tests * chore: concurrent copying * chore: cosmetic changes * chore: missed an import * chore: checkpoint 1 – remove #[serial] for most client tests * chore: add TODOs for later * chore: don't retry in CI; flaky tests are bugs to be fixed. * chore: fix threshold tests * chore: added todos * chore: return Result for nicer backtraces and consistency * chore: root out more serial tests * Add a todo about `integration-serial` test group. * chore: formatting * chore: tighten up comments, imports and features used * chore: ditch `working-directory` from main.yml job * chore: add TODO about possibly deprecating the single redis-needing test in the workspace * chore: don't run tests for `experiments` three times and avoid using redis where we don't need to. * chore: add TODO about retiring kms-custodian binary check for tests * chore: update TODOs * chore: remove common-testing-big-instance – UNUSED * chore: workflow cleanup: - Remove stale env vars&inputs - gate redis on `run-redis` via a step, not a job-level `services:` - remove steps we do not use (docs&npm-release) - avoid setting defaults and CWD * chore: modernize LFS install * chore: no more funny url rewriting ssh urls injecting credentials * chore: more CI workflow cleanup - Restore test-core-client-docker-tls - Build prepare-*-matrix outputs with `jq -nc` instead of escaped string concat — output is byte-identical, but readable - Tighten actions/checkout in common-testing.yml, rust-lint.yml, and the docker-tls job: drop `submodules: true` - Drop now-orphaned BLOCKCHAIN_ACTIONS_TOKEN - Fold workspace-utils matrix entry into crates-normies * chore: outcome of analysing the lists of tests ran by each job to avoid duplicates. * chore: add TODO about unmaintainability of manual splitting of the test work. * fix: multiple nextest process race to write key info to disk and may clobber each other's data. * chore: fix up imports * fix: is 1.95 to blame for slow compilation? * fix: wasm-tests * chore: merge fixes * chore: remove version pinning. We set the version elsewhere. chore: remove the integration-serial test group chore: remove config-file filtering, we prefer explicit filters * chore: make the two last tests work in a concurrent setting * chore: tweak comments a bit * chore: remove git-lfs install chore: disentangle skip-test-material from LFS usage * chore: formatting * chore: fix missing lfs * chore: remove the serial_test crate from most places * chore: re-think the three CI config values that touch shared testing material. Remove all facilities for shared testing material. All tests are now isolated, so also remove the `force_isolated()` method and associated code. Test groups that make copies of generated test material now generate both testing/production-like material. * formatting * chore: update stale comments, don't run helm&docs jobs on toolchain changes * choreÖ update the CI README * chore: remove stale TODO * chore: cleanup unused test helpers * chore: attempt to ensure that storage variability does not cause flaky tests * chore: remove serial_test from the workspace * chore: a couple of cosmetic fixes * chore: cleanup stale todos * chore: missed on material generation command * chore: co-pilot review feedback fixes
Description of changes
This moves all experiments and bechmarks (mostly needed for NIST) binaries into the old
threshold-fhe(nowthreshold-experimentcrate).Also removes dependancy to
threshold-fhe/experimentsof all other crates, as it's now a "binary only" crate (i.e. not a lib anymore).Wondering whether
threshold-experimentsis the best name, especially since there are non-threshold test/kats/benches in it ...Maybe just
experiments?Also thinking about renaming
threshold-experimentaltothreshold-bgvcause I find experimental a bit vague (but then my issue is that there's no explicitthreshold-tfhecrate as it's part ofthreshold-execution...).Also, features are a mess.
Issue ticket number and link
Closes https://github.com/zama-ai/kms-internal/issues/2973
PR Checklist
I attest that all checked items are satisfied. Any deviation is clearly justified above.
chore: ...).TODO(#issue).unwrap/expect/paniconly in tests or for invariant bugs (documented if present).devopslabel + infra notified + infra-team reviewer assigned.!and affected teams notified.Zeroize+ZeroizeOnDropimplemented.unsafe; if unavoidable: minimal, justified, documented, and test/fuzz covered.Dependency Update Questionnaire (only if deps changed or added)
Answer in the
Cargo.tomlnext to the dependency (or here if updating):More details and explanations for the checklist and dependency updates can be found in CONTRIBUTING.md