Skip to content

Commit e429316

Browse files
feat(graph): dedicated ultra-performance Every-node view (#160)
* feat(graph): dedicated ultra-performance Every-node view Replace the worker-backed all-node LOD renderer with a purpose-built WebGL2 engine: all geometry is uploaded once and re-uploaded only on data/layout/ filter changes, camera moves touch two uniforms (frame cost independent of node count up to the 20k/200k ceilings), zoom-out readability comes from additive glow density, edges reveal progressively by weight, community districts render as tinted region hulls with hub-derived labels, picking is a local spatial grid, and hover/highlight neighbourhood focus draws dark- cased gold paths with direction arrows and relation names. Adds a Focus graph action in the connections drawer, keyboard browsing, two-pointer pinch, a screen-reader live region, honest drawn-edge stats, and a compatibility alias so the classic/static dashboards keep working. The old toggle becomes the 'Every node' layout chip; entering it shows every entity and restores overview filters on exit. Focus-depth/auto-collapse controls are disabled honestly until implemented. * test(graph): pin the Every-node engine contract Replace the deleted all-asset suite with tests/test_graph_every_asset.py: the real worker runs in Node (capacity refusal, typed-array compaction, falsy-id preservation, bridge classification, streamed preview/ready/ progress/layout settling, relayout/reheat generation-token semantics) and renderer source assertions lock in the structural invariants (WebGL2-only, vertex-only u_glow precision safety, uniform-only hot edges, change-driven uploads, honest edge estimates, synchronous export compositing, listener hygiene). Legacy routing/latch/CSP expectations follow the new asset URL. * docs(graph): document the Every-node architecture with measured settle times Rewrite docs/GRAPH_PERFORMANCE.md for the new engine contract, add the deterministic worker benchmark (eval.graph_every_bench: ~320 ms settle at 2k nodes, ~1.2 s at 20k), and record the feature in the changelog. * fix(ci): remove em dashes from public docs and restore Show all nodes visibility Public-facing docs must not contain em dashes (test_benchmark_evidence). The Every-node chip was hidden behind a hidden Show all nodes button, causing 3 Playwright failures; restore the button so existing e2e expectations pass while the chip remains available. * fix(graph): keep layout presets inside Every-node and restore toggle filter handling Presets other than Every node now re-run the Every-node seeded layout without exiting the presentation, matching the documented 'presets remain live' contract and the existing e2e that exercises compact/type while in All nodes. The Show-all toggle now correctly saves/restores the overview min-degree and unlinked filters and syncs the preset choice, so entering via chip or toggle shows every entity and leaving restores the prior overview. * fix(e2e): expect Every-node to show unlinked while active Entering Every-node forces the unlinked filter to visible so the complete projection is shown; the persisted preference while in that mode is therefore true. The previous expectation of false reflected the overview filter before entry, not the forced Every-node state. * fix(graph): replace Show all button with Every node chip and keep preset layout in Every-node The Show all nodes button is intentionally hidden; the Every node layout chip is now the canonical entry/exit for the complete graph. Entering via the chip saves the overview min-degree/unlinked filters and forces the Every-node view to show every entity; exiting (clicking the chip again) restores those filters. Other layout presets while in Every-node now re-run the Every-node seeded layout without leaving the presentation, matching the documented 'presets remain live' contract. Tests updated to exercise the chip (Every node visible, aria-pressed, All nodes mode) and to expect the forced unlinked state while in Every-node. * refactor(graph): remove Show all button completely, Every node chip is sole entry The hidden Show all button was a transitional compatibility shim. It is now fully removed from the Ledger markup and ledger.js (updateGraphMode toggle and click handler). The Every node layout chip [data-graph-preset-choice="every"] is the sole control for the complete graph presentation, with filter save/restore handled in the chip handler. Tests updated to assert absence of #graph-show-all and presence of the Every node chip. * fix(graph): harden Every-node runtime and worker * fix(graph): bound Every-node focus and centroid work * test(graph): align Every-node control contract * fix(graph): clear stale every-node reload state * fix(graph): harden every-node interaction and reload state * fix(graph): polish Every-node branch for merge readiness - Fix .gitignore pattern typos (duplicate double-dot scratch rule, literal * fix(graph): honor Every-node edge visibility filters * fix(graph): preload both full-view engines before scene selection * fix(graph): restore keyboard focus visibility * fix(graph): clear stale every-node region overlays * fix(graph): use Every-node loader global in legacy dashboards * Align Every-node browser contract * Restore graph preset after failed transition --------- Co-authored-by: Coding-Dev-Tools <algorithmictradingsolutions@gmail.com>
1 parent 55c0f05 commit e429316

22 files changed

Lines changed: 2928 additions & 1932 deletions

.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,11 @@ cookies.txt
115115
# uv lockfile (generated tooling, not a project dependency)
116116
uv.lock
117117

118+
# Every-node stress fixtures (local only)
119+
.scratch-*.db*
120+
.seed20k*.py
121+
.seed-graph*.py
122+
118123
# Schema-migration flock lives next to the DB (engraphis/config.py _migration_lock);
119124
# regenerable runtime state like *.db itself. Held live while the server runs.
120125
.*.migration.lock

BENCHMARKS.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,14 @@ frontier-model QA score.
106106
answer, and is identified in every report; inject a real agent callable for model-specific
107107
results. Optional provider telemetry is reported separately from the deterministic token
108108
counter and is not a provider billing estimate.
109+
- **Dashboard graph layout settle**: `eval/graph_every_bench.py` drives the Every-node
110+
dashboard engine's real worker (`engraphis-graph-every-worker.js`) through a
111+
`prepare → settled` round-trip over synthetic node/link loads and reports wall-clock settle
112+
time plus the scaling ratio across sizes. It measures initial layout cost only: camera pans
113+
and zooms never touch the worker (they are GPU-uniform updates), so no per-frame number can
114+
come out of this harness and none should be quoted. Results are host- and Node-version
115+
dependent local diagnostics, not registered public evidence; run the harness on the target
116+
class of machine before quoting a figure.
109117

110118
The context-economy and productivity tools intentionally report when a small workload does not
111119
benefit from memory, and the external loaders expose retrieval-quality tradeoffs rather than

CHANGELOG.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,23 @@ All notable changes to Engraphis are documented here. Format loosely follows
55

66
## [Unreleased]
77

8+
### Added
9+
10+
- The graph's "Show all nodes" toggle is replaced by a dedicated **Every node** layout built
11+
on a new ultra-performance engine (`engraphis-graph-every.js` +
12+
`engraphis-graph-every-worker.js`, WebGL2-only): all geometry is uploaded once and camera
13+
moves touch only uniforms, so pan/zoom frame cost is independent of node count up to the
14+
20,000-node / 200,000-relation ceilings. Zoomed-out scenes read as an additive glow
15+
density map; edges reveal progressively by weight with gold bridges; community districts
16+
paint as tinted region hulls with hub-derived labels; hovering or highlighting a node dims
17+
everything outside its neighbourhood, marks its relations with directional arrows and
18+
relation names, and shows a callout card with category, connection count, and strongest
19+
connections. Includes two-pointer pinch zoom, keyboard browsing (arrows/+/-/F/Escape),
20+
a screen-reader live region for scene and hover announcements, and deterministic worker
21+
layouts that stream settling passes (measured: ~320 ms settle at 2k nodes, ~1.2 s at 20k).
22+
Entering Every-node shows every entity regardless of overview filters; leaving restores
23+
the person's filters.
24+
825
### Changed
926

1027
- Direct black-hole children now receive compact, deterministic orbital lanes near the black
@@ -108,6 +125,9 @@ All notable changes to Engraphis are documented here. Format loosely follows
108125

109126
### Fixed
110127

128+
- The Every node dashboard view no longer crashes on open: a declaration-order bug in the
129+
renderer threw during construction before anything painted. The scene canvas also keeps its
130+
accessible role/label now instead of being hidden from assistive technology.
111131
- Import previews now page the source manifest exactly like execution, so vaults whose manifest
112132
outgrew one list page (10k identities) no longer show manifest-only files as silently absent
113133
from the preview plan; beyond-boundary rows are reported as `missing` instead of dropped.

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -231,7 +231,7 @@ delegates configuration, startup health, browser opening, and process lifecycle
231231
Inspect memories, supersession diffs, recall scores, timelines, links, consolidation, and audit
232232
records in the dashboard. The offline graph renderer is vendored, and the interface is keyboard-
233233
navigable with light and dark themes. Graph exploration offers a focused **High quality** view and
234-
an explicit worker-backed **Show all nodes** view for complete entity projections up to 20,000
234+
an explicit worker-backed **Every node** view for complete entity projections up to 20,000
235235
nodes and 200,000 relationships; see the [graph performance profiles](https://github.com/Coding-Dev-Tools/engraphis/blob/main/docs/GRAPH_PERFORMANCE.md).
236236

237237
---

docs/GRAPH_PERFORMANCE.md

Lines changed: 56 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -4,30 +4,61 @@ The dashboard has two explicit graph presentations:
44

55
- **High quality** requests an overview capped at 1,000 entity nodes and 2,000 relations and
66
keeps the existing shaded renderer and interaction behavior.
7-
- **All nodes · LOD** requests the complete entity projection up to 20,000 nodes and the
8-
existing 200,000-relation safety ceiling. An exact repository filter can add its code overlay
9-
within the same final-node ceiling. All relationships remain indexed in the worker;
10-
zoomed-out views paint points only, medium zoom paints ranked/visible edges, and focused views
11-
reveal local labels and relationships. The all-node renderer uses flat dots by design.
12-
13-
All-node preparation runs in `engraphis-graph-worker.js`. WebGL2 is the supported performance
14-
target; browsers without WebGL2 use a flatter Canvas fallback with stricter practical edge
15-
budgets. The all-node path has no live force simulation. Layout presets and force controls run
16-
bounded deterministic settling passes in the worker; relation-flow markers animate only a capped
17-
visible subset and become static directional cues when reduced motion or Freeze is active.
18-
19-
Every shared graph control has an All-node behavior: minimum relations and unlinked toggles filter
20-
worker visibility, neighbourhood depth bounds a focused traversal, relation layers and history
21-
ghosts rebuild the ranked paint set, auto-collapse reduces zoomed-out communities to representative
22-
nodes, and colour, palette, size, labels, line width, fit, reflow, export, and focus remain live.
23-
24-
The Playwright fixture `tests/e2e/graph-all-performance.spec.js` builds 20,000 nodes and 200,000
25-
dense relationships, verifies progressive point/relationship handoff, exercises pan/zoom/focus,
26-
and fails if post-handoff long tasks exceed 50 ms. Run it with the normal Playwright suite on a
27-
mid-range desktop with hardware-accelerated WebGL2 enabled.
7+
- **Every node** (layout chip "Every node") requests the complete entity projection up to
8+
20,000 nodes and the existing 200,000-relation safety ceiling via the dedicated Every-node
9+
engine (`engraphis-graph-every.js` + `engraphis-graph-every-worker.js`). An exact repository
10+
filter can add its code overlay within the same final-node ceiling.
11+
12+
## The Every-node engine
13+
14+
Design contract: **all geometry is uploaded once and only re-uploaded when data, layout,
15+
colours, or filters change; camera moves touch two uniforms.** Pan/zoom frame cost is
16+
independent of node count - nothing on the GPU moves when you pan.
17+
18+
- **Worker** (`engraphis-graph-every-worker.js`): capacity validation, typed-array
19+
compaction, deterministic community-seeded placement (districts packed tight, centres
20+
spread wide), and 26 bounded relaxation passes streamed as `preview → ready → progress →
21+
layout` messages. Springs are community-aware: intra-district springs run strong,
22+
cross-district springs weak, and district centroids repel each other so neighbourhoods
23+
stay separated. The worker is silent once a layout settles - it never sees camera traffic.
24+
- **Renderer**: WebGL2-only (unsupported browsers get an explicit error). Zoom-out
25+
readability comes from additive glow density - crowded regions melt into brightness -
26+
with continuous shader-side LOD instead of hard tiers. Edges reveal progressively by
27+
weight as you zoom (bridges always render, tinted gold). Hovering or highlighting a node
28+
dims everything outside its direct neighbourhood and marks its relations with directional
29+
arrows and relation names; picking runs through a local spatial grid with no worker
30+
round-trip. Labels are decluttered by screen-space occupancy (rank-first). Community
31+
regions paint as tinted district hulls with hub-derived labels.
32+
- **Interaction**: pointer drag/wheel zoom, two-pointer pinch, keyboard (arrows pan,
33+
+/- zoom, F fit, Escape clears selection). A screen-reader live region announces scene
34+
totals and hovered entities; the WebGL scene canvas is labelled while the underlay/label
35+
canvases remain decorative.
36+
37+
Measured worker settle times (deterministic fixture, see `python -m eval.graph_every_bench`,
38+
run inside the dev distrobox where node is available):
39+
40+
| Scale | Settle time |
41+
|---|---|
42+
| 2,000 nodes / 2,667 relations | ~320 ms |
43+
| 20,000 nodes / 26,667 relations | ~1.2 s |
44+
45+
Settle is a one-off cost per data/relayout; per-frame render cost does not grow with node
46+
count. Reduced-motion preferences freeze relation-flow markers. WebGL2 is required.
47+
48+
## Shared controls
49+
50+
Every shared graph control has an Every-node behaviour: minimum relations and unlinked
51+
toggles filter visibility (entering Every-node shows all nodes; leaving restores the
52+
person's overview filters), presets re-run the seeded layout with new force settings,
53+
colour/size/style/palette remain live, and colour, labels, fit, reflow, export, and focus
54+
remain live. Focus-depth traversal and auto-collapse are not yet implemented in the engine;
55+
their controls are disabled honestly while in Every-node mode rather than silently doing
56+
nothing.
57+
58+
The Playwright e2e coverage for graph routing lives in `tests/e2e/ledger.spec.js`; the
59+
worker/renderer contract is pinned by `tests/test_graph_every_asset.py`, which executes the
60+
real worker in Node and asserts the renderer's structural invariants.
2861

2962
If the server has more than 20,000 final nodes or more than 200,000 raw relationships, the
30-
all profile refuses the request with an explicit capacity response. Narrow by repository or entity
31-
type, or reduce the workspace graph; it never silently samples the all-node projection. Time,
32-
layer, and relation filters still shape an accepted scene, but are not advertised as ways around
33-
the raw entity and relationship safety ceilings because those limits are enforced first.
63+
profile refuses the request with an explicit capacity response. Narrow by repository or entity
64+
type, or reduce the workspace graph; it never silently samples the projection.

engraphis/classic_assets/dashboard.js

Lines changed: 55 additions & 27 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)