Skip to content

Phase B: per-phase memory instrumentation (ADR-029)#27

Merged
chaospheremk merged 2 commits into
mainfrom
chore/memory-instrumentation
May 18, 2026
Merged

Phase B: per-phase memory instrumentation (ADR-029)#27
chaospheremk merged 2 commits into
mainfrom
chore/memory-instrumentation

Conversation

@chaospheremk

Copy link
Copy Markdown
Owner

Phase B — Memory instrumentation

Phase B of the reconciliation plan (plans/polished-inventing-whisper.md). PR #26 ported six categories of work-computer platform fixes onto trunk, but the underlying 8+ GB working-set ceiling on the ~30k-object live run is undiagnosed (no JSONL retained). Before another live run, add structured memory measurement so the next run produces data we can reason from.

What changed

scripts/Invoke-ADPermissionAnalysis.ps1

  • New Get-RuntimeMemorySnapshot helper. Returns a 6-field hashtable:
    • managedHeapMBGC.GetTotalMemory($false) / 1MB, 1dp (no forced collection)
    • workingSetMBProcess.WorkingSet64 / 1MB, 1dp
    • privateBytesMBProcess.PrivateMemorySize64 / 1MB, 1dp
    • gen0Collections, gen1Collections, gen2CollectionsGC.CollectionCount(n)
  • Snapshot embedded in data.memory for:
    • Every PhaseStart / PhaseEnd event (Phases 1-6, 12 boundaries).
    • Every Phase 2 EnumerationProgress event (~5000-object cadence).
    • New MidEnumerationDrain events emitted only when a drain pass freed >=1 completed handle (so post-run analysis can correlate drain bursts with managed-heap movement).

The helper lives in the entry script. Runspaces don't call it — the orchestrator's drain block already sees per-batch boundaries.

CHANGELOG.md

Entry under [Unreleased] / Added.

Why this and not a speculative memory fix

Phase A already shipped the obvious memory mitigations (Item-pinning removal, mid-enumeration drain, incremental $aceRecords). Those weren't enough. Without per-phase telemetry we don't know:

  • Whether the peak lives in Phase 2 (in-flight pipelines), Phase 3 (parsed ACE list), Phase 4 (caches), Phase 5 (index), or Phase 6 (pivot accumulator).
  • Whether the heap is managed-side dominant or process-side (unmanaged buffers, AD assembly internals).
  • Whether GC is collecting at all (gen-2 count tells us).

Speculate-fixing into that unknown is how engineering teams build the wrong solution well. Measure first.

ADR-029 in the vault project notebook captures the rationale.

Verification

  • Invoke-Build -Configuration Release — clean, 150 tests pass / 0 failed / 1 skipped, coverage 93.01% (gate 86% per ADR-026).
  • Hand run against unreachable.invalid (LDAP bind fails as expected, exit 1) — first PhaseStart event in the JSONL carries data.memory with all 6 fields populated.

Follow-up

Phase C of the reconciliation plan: controlled re-run protocol on the work computer using this instrumentation. Phase D selects targeted optimisations from the captured data.

Notes

  • No behaviour change. Helper is read-only — no GC.Collect, no Process.Refresh, no side effects.
  • No new dependencies. System.GC and System.Diagnostics.Process are already loaded.
  • The ADR lives in the vault project notebook (../decisions.md) under the dual-track workflow, not in this repo.

…vents

Add Get-RuntimeMemorySnapshot helper returning managedHeapMB,
workingSetMB, privateBytesMB (all 1dp), and gen0/1/2 collection counts.
GC.GetTotalMemory($false) so no GC pass is forced.

Snapshot embedded in data.memory for:
  - Every PhaseStart and PhaseEnd event (Phases 1-6 = 12 boundaries).
  - Every Phase 2 EnumerationProgress event (~5000-object cadence).
  - New MidEnumerationDrain events emitted only when a productive
    drain pass freed >=1 completed handle, so post-run analysis can
    correlate drain bursts with managed-heap movement.

Cheap: no allocations beyond the returned hashtable, no I/O. Produces
a runtime memory time-series in the JSONL log without changing
script behavior. Prepares for the next live-environment diagnostic run
against the work-computer 30k-object domain that hit a memory ceiling
even after the Phase A drain/Item-pinning fixes.

ADR-029 captures the rationale.
Records Get-RuntimeMemorySnapshot helper + the events it rides on
(12 phase boundaries + Phase 2 EnumerationProgress + new
MidEnumerationDrain). The ADR itself lives in the vault project
notebook at ../decisions.md.
@chaospheremk
chaospheremk merged commit 0b0d2f7 into main May 18, 2026
6 checks passed
@chaospheremk
chaospheremk deleted the chore/memory-instrumentation branch May 18, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants