perf: optimize directory reconstruction in recursive find/walk

[raj-prince]

Summary
Optimizes GCSFileSystem._get_dirs_and_update_cache (used in find(), walk(), and recursive listings) by creating directory metadata dictionaries only once per unique folder and short-circuiting ancestor traversal when a directory branch is already registered.

Reduces time complexity from $\mathcal{O}(N \times \text{Depth})$ to $\mathcal{O}(N + U \times \text{Depth})$ (N: total files, U: unique directories), yielding up to a 14.2x speedup (93% less CPU time) and eliminating >99% of heap dictionary allocations.

Benchmark Results (similar to gcsfs/tests/perf/microbenchmarks/ but hierarchy generated in memory - no network calls)

Scenario (from configs.yaml)	Geometry	Legacy	Optimized	Speedup
find_flat_folders	65k files, 256 folders, depth 0	73.54 ms	28.66 ms	2.57x (61% less CPU)
find_recursive_folders	65k files, 256 folders, depth 8	191.11 ms	30.16 ms	6.34x (84% less CPU)
find_recursive_folders_deep	65k files, 256 folders, depth 24	458.42 ms	33.48 ms	13.69x (93% less CPU)
find_recursive_folders_deep	131k files, 256 folders, depth 24	940.39 ms	65.98 ms	14.25x (93% less CPU)

Changes

1. Allocate Directory Dicts Once: Check parent_already_seen = parent in dirs to avoid recreating duplicate dicts for every sibling file.
2. Short-Circuit Visited Ancestors: Break early once a child is in listing and its parent's ancestors are established.
3. Immediate Break on Prefix/No-Cache: Exit on iteration 1 for already-seen folders when prefix is set or update_cache=False.
4. Pre-computed Variables: Calculate root_len and should_update_cache outside loops.

Testing

• Passes all existing tests in test_core.py and test_extended_hns.py.
• Pre-commit (black, flake8, isort) clean.

[gemini-code-assist]

Code Review

This pull request optimizes _get_dirs_and_update_cache in gcsfs/core.py by caching the root path length and avoiding redundant directory key splits and cache updates when a parent directory has already been processed. It also adds unit tests to verify the nested directory cache behavior. The reviewer suggested a further optimization to immediately break the loop when a parent has already been seen and either a prefix is set or cache updating is disabled, avoiding redundant ancestor traversals.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.20%. Comparing base (d4161bf) to head (996b652).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1046      +/-   ##
==========================================
+ Coverage   90.18%   90.20%   +0.01%     
==========================================
  Files          16       16              
  Lines        3740     3747       +7     
==========================================
+ Hits         3373     3380       +7     
  Misses        367      367              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.