feat(perf): add microbenchmark suite for cat_ranges

[yuxin00j]

Overview

This PR adds a comprehensive microbenchmark suite for cat_ranges under gcsfs/tests/perf/microbenchmarks/cat_ranges. It benchmarks multi-range read throughput and latency across Regional, Zonal (Rapid), and Hierarchical Namespace (HNS) buckets under various access patterns and parameter combinations.

Summary of Changes

1. Benchmark Suite (gcsfs/tests/perf/microbenchmarks/cat_ranges/):

   • parameters.py: Defines CatRangesBenchmarkParameters capturing range counts, range selection patterns, chunk sizes, max_gap, and batch_size.
   • configs.py: Implements CatRangesConfigurator to generate test matrices from YAML configurations.
   • configs.yaml: Defines baseline scenarios covering:
     • cat_ranges_seq: Sequential adjacent ranges across a file.
     • cat_ranges_rand: Uniform random ranges across a file.
     • cat_ranges_mixed_sizes: Variable chunk sizes (64 KB, 1 MB, 4 MB, 16 MB) within the same call.
     • cat_ranges_multi_file: Multi-file range requests distributed round-robin across 5 files.
     • cat_ranges_batch_size_sweep: Concurrency sweep evaluating batch sizes 16, 64, and 128.
     • cat_ranges_large_file_large_ranges: Scale scenario requesting 1,000 mixed ranges across a 10 GB file with batch size 64.
   • test_cat_ranges.py: Implements test_cat_ranges_single_threaded benchmark runner using pytest-benchmark and pre-generated range offsets to avoid measuring setup overhead.

2. Fixtures and Metrics Reporting (gcsfs/tests/perf/microbenchmarks/conftest.py):

   • Added gcsfs_benchmark_cat_ranges fixture to manage lifecycle of temporary test files across bucket types.
   • Enhanced publish_benchmark_extra_info to record num_ranges, max_gap, and batch_size in the benchmark report and output CSVs.

3. Test Framework Registration (gcsfs/tests/conftest.py):

   • Registered cat_ranges in benchmark_subdirs to ensure test collection when passing --run-benchmarks.

4. Unit Test Coverage (gcsfs/tests/perf/microbenchmarks/test_configs.py):

   • Added test_cat_ranges_configurator unit test.
   • Registered get_cat_ranges_benchmark_cases() in test_validate_actual_yaml_configs to validate YAML integrity in CI.

[gemini-code-assist]

Code Review

This pull request introduces a new microbenchmark suite for the cat_ranges operation in gcsfs, including configuration setups, benchmark parameters, and single-threaded test cases. The review feedback highlights several critical improvement opportunities: correcting the sequential range generation logic to prevent backward jumps, adding validation to ensure range sizes do not exceed file sizes, and robustly handling potentially empty or null configuration values for chunk sizes, batch sizes, and maximum gaps to avoid runtime errors.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.08%. Comparing base (8a3d3f5) to head (78c3c36).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1035      +/-   ##
==========================================
- Coverage   90.10%   90.08%   -0.02%     
==========================================
  Files          16       16              
  Lines        3679     3683       +4     
==========================================
+ Hits         3315     3318       +3     
- Misses        364      365       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Yonghui-Lee]

Throughput calculation in run.py is wrong for cat_ranges. For cat_ranges, the actual data transferred is not file_size * files.

[yuxin00j]

Throughput calculation in run.py is wrong for cat_ranges. For cat_ranges, the actual data transferred is not file_size * files.

Done. Updated the throughput calculation logic to use total_bytes.