refactor: replace non-dominated sorter implementations with ndsort - #63
Merged
Conversation
foolnotion
changed the title
…nd SIMD
Replace the old cppsort-based RankIntersect implementation with a
ground-up rewrite optimised for NSGA-II workloads:
- LSB radix sort (11-bit passes) with IEEE 754 float→sortable-uint
transformation, avoiding any cppsort dependency
- Structure-of-Arrays fitness layout (fvals[obj * n + i]) for
sequential memory access during objective passes
- PackedPool: triangular bitset allocator giving individual j exactly
(nb - j/DIGITS) words; total footprint ≈ n²/128 words (~16 MB at n=16k)
- SIMD bitset intersection via eve::algo::transform_to with std::bit_and
- Early-exit in ObjectiveLoop: tracks a `done` counter and breaks out
of the objective loop once all individuals have empty domination sets
Also add infrastructure used during development/benchmarking:
- GeneticAlgorithmBase::SortTime() — cumulative non-dominated sort time
instrumented in NSGA2::Sort()
- --sorter rs|ms CLI flag for operon_nsgp to select sorter at runtime
- sort_ms column in reporter output
Benchmarks on Poly-10 (n=500, 16 threads, 1M evaluations, seed=42):
pop=500: old RS 271ms / new RS 186ms / MS 524ms (1.5× vs old, 2.8× vs MS)
pop=1000: old RS 321ms / new RS 223ms / MS 776ms (1.4× vs old, 3.5× vs MS)
pop=2000: old RS 481ms / new RS 277ms / MS 1270ms (1.7× vs old, 4.6× vs MS)
pop=5000: old RS 1080ms / new RS 502ms / MS 3400ms (2.2× vs old, 6.8× vs MS)
Correctness verified: all 14 [algorithms] tests pass; 200-seed sweep
confirms exact agreement with DeductiveSorter across n={100,1000,5000},
m={2,3,4,5,10}.
Adds heal-research/ndsort as a flake input (nixpkgs.follows wired), injects it via its overlay, and lists it in operon.nix buildInputs.
Removes 8 hand-rolled sorter .cpp files (1034 lines) and replaces them with a 29-line adapter over the ndsort library, which provides all nine algorithms through a uniform, well-tested interface. Key changes: - Add ndsort as a flake input (git+ssh://github.com/foolnotion/ndsort) and wire it into CMakeLists.txt via find_package + target_link_libraries - Delete: best_order_sort, deductive_sort, dominance_degree_sort, efficient_sort, hierarchical_sort, merge_sort, rank_intersect, rank_ordinal .cpp files - Add ndsort.cpp: thin Wrap<S> template calling each ndsort sorter with the sorted_unique tag, skipping redundant lex-sort and eps-dedup since NSGA2::Sort already partitions out duplicates before calling the sorter - Remove EfficientSortStrategy enum (was only used internally) - Fix float preservation in flatten(): use conditional_t instead of common_type_t to avoid silently promoting float fitness to double - Fix sort_ms column missing from compare_operon.py _STAT_COLS Validated: 72/72 determinism checks pass (bit-identical output vs ref), 24/24 statistical equivalence groups pass at α=0.01 across four datasets and both operon_gp and operon_nsgp.
foolnotion
force-pushed
the
feat/rank-intersect-sorter-rewrite
branch
from
May 26, 2026 06:13
0777a75 to
01ad6a9
Compare
foolnotion
added a commit
that referenced
this pull request
Jul 13, 2026
…rewrite refactor: replace non-dominated sorter implementations with ndsort
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ndsortlibrary, which provides all nine algorithms through a uniform, well-tested interfacesorted_uniquetag to skip redundant preprocessing:NSGA2::Sortalready lex-sorts and partitions out duplicates before calling the sorter, so ndsort's internal lex-sort andeps_deduppasses are bypassed entirelyflatten():std::common_type_t<float, double>was silently promoting single-precision fitness values todouble, causing 6 radix passes instead of 4; fixed withstd::conditional_tsort_mscolumn missing fromcompare_operon.py_STAT_COLS, which was causingelapsedto be read assort_msValidation
operon_gpandoperon_nsgpTest plan
operon_nsgpruns with each--sorterflag