Skip to content

feat: reference-parity swap_or_not_shuffle API and JS binding - #559

Open
nazarhussain wants to merge 12 commits into
mainfrom
nh/swap-or-not-shuffle-parity
Open

feat: reference-parity swap_or_not_shuffle API and JS binding#559
nazarhussain wants to merge 12 commits into
mainfrom
nh/swap-or-not-shuffle-parity

Conversation

@nazarhussain

@nazarhussain nazarhussain commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Motivation

Stacked on #558. Ports the full @chainsafe/swap-or-not-shuffle API into lodestar-z so Lodestar can consume it from the zig bindings — the surface ChainSafe/lodestar#9263 uses. Zero behavioral divergence from the Rust package, with performance as the priority.

Description

Module (src/swap_or_not_shuffle) — line-faithful port of the reference lib.rs:

  • innerShuffleList/shuffleList/unshuffleList with the reference's exact validation order (rounds==0 and len<=1 return before seed validation; seed errors take precedence over rounds errors) and error set; generic over element type so state_transition keeps using u64
  • ComputeShuffledIndex with per-round pivot/source caching
  • getCommitteeIndices, computeProposerIndex(Electra), computeSyncCommitteeIndices(Electra), ByteCount, SHUFFLE_ROUNDS_MAINNET/MINIMAL

Binding (bindings.shuffle) — identical names, argument order, return values, and error messages as the npm package:

  • sync shuffleList/unshuffleList return a new Uint32Array without mutating the input
  • asyncShuffleList/asyncUnshuffleList run on the libuv pool and resolve zero-copy via an external ArrayBuffer
  • ComputeShuffledIndex class, the four committee-index functions, constants + ByteCount enum object
  • Reference error strings verbatim (e.g. "Shuffling seed must be 32 bytes long") for sync throws and async rejections

Zero-divergence validation

  • bindings/test/shuffle.test.ts ports the reference repo's own test suite, using an in-repo pure-TS reference implementation (bindings/test/shuffleReference.ts, deps inlined) as the oracle on random seeds — no dependency on the npm package. Out-of-range wrapping vectors are hardcoded from @chainsafe/swap-or-not-shuffle v1.2.1 output
  • 2M indices × 90 rounds × 3 random seeds sweep (shuffle + unshuffle): zero divergence
  • Zig unit tests carry the reference repo's fixed vectors (spec vector, proposer/sync-committee vectors)

Performance (darwin-arm64)

One-off comparison against the Rust npm package (v1.2.1) during development:

Benchmark zig rust
unshuffleList 100k 15.2 ms 19.9 ms
unshuffleList 2M 332 ms 318 ms (parity within noise; ReleaseFast: 301 vs 312 ms)
computeSyncCommitteeIndicesElectra 1M 5.4 ms 6.9 ms
computeProposerIndexElectra 1M 14.9 µs 24.9 µs

Note: pnpm prepare builds bindings with ReleaseSafe, which leaves the big-list shuffle ~4% behind the Rust package; ReleaseFast beats it. Left the project default untouched — worth a separate discussion.

Out of scope

computePtcIndices/computePtcIndicesForEpoch (the new functions ChainSafe/lodestar#9263 also needs) are still unmerged upstream (ChainSafe/swap-or-not-shuffle#24); they'll come as a follow-up once upstream settles, ported the same way.

🤖 Generated with Claude Code

Base automatically changed from nh/swap-or-not-shuffle to main August 13, 2026 11:23
Port the full @chainsafe/swap-or-not-shuffle surface into the module
(exact validation order and errors, ComputeShuffledIndex, proposer and
sync committee index functions) and expose it as bindings.swapOrNotShuffle
with reference-identical names, semantics, and error messages.

Differential tests compare every function against the npm package;
2M-index sweeps show zero divergence. Perf benches show equal or faster
results than the Rust binding. Needed for ChainSafe/lodestar#9263.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@nazarhussain
nazarhussain force-pushed the nh/swap-or-not-shuffle-parity branch from 71ec60d to 01edd6b Compare August 13, 2026 11:26
nazarhussain and others added 8 commits August 13, 2026 16:36
Everything now lives under bindings.shuffle: the reference-parity API
plus the pre-existing in-place innerShuffleList variant. Drops the
separate swap_or_not_shuffle.zig binding file and namespace.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
bindings/test/swapOrNotShuffle.test.ts merges into shuffle.test.ts and
bindings/perf/swapOrNotShuffle.test.ts becomes perf/shuffle.test.ts,
matching bindings/napi/shuffle.zig.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Confines the raw napi async-work/Deferred plumbing to one reusable
module. Upstream candidate for a zapi js.AsyncTask; transferOwnedSlice
is superseded by zapi#68 OwnedTypedArray once released.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
shuffle.zig is now zapi DSL plus a small ShuffleTask; all raw napi
plumbing lives in async_task.zig. No JS-visible behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ComputeShuffledIndex.get and the committee weighting comparison now use
wrapping ops matching the Rust reference's release-mode behavior, so
out-of-range JS inputs produce reference-identical values instead of
aborting the process. Also fixes ByteCount d.ts typing, adds error-path
tests, and refreshes stale comments.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tation

Ports the reference repo's test suite and its pure-TS reference
implementation (deps inlined: sha256 via node:crypto, spec constants,
naive computeShuffledIndex/committee sampling) so the shuffle tests are
self-contained. Out-of-range wrapping vectors are hardcoded from
@chainsafe/swap-or-not-shuffle v1.2.1 output. Drops that package as a
devDependency; it remains only as a transitive dep of
@lodestar/state-transition.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The zapi DSL only auto-exports functions and classes; rather than a
native register hook, the loader attaches the three reference constants
to the shuffle namespace. Can move back into Zig once zapi auto-exports
namespace-level consts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread bindings/src/bindings.js
Comment on lines +6 to +11
// Namespace-level constants mirroring @chainsafe/swap-or-not-shuffle; the
// zapi DSL only auto-exports functions and classes, so values are attached
// here instead of via a native register hook.
bindings.shuffle.SHUFFLE_ROUNDS_MAINNET = 90;
bindings.shuffle.SHUFFLE_ROUNDS_MINIMAL = 10;
bindings.shuffle.ByteCount = {One: 1, Two: 2};

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will soon go away as we will support namepsace pub const and enum in the Zapi.

nazarhussain and others added 2 commits August 14, 2026 11:19
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
package.json is net-unchanged on this branch (devDependency added then
removed), so the lockfile drift from those installs (refreshed
deprecation metadata, re-resolved sax) does not belong in the diff.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@nazarhussain
nazarhussain marked this pull request as ready for review August 14, 2026 06:53
@nazarhussain
nazarhussain requested a review from a team as a code owner August 14, 2026 06:53
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant