Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
285 lines (199 loc) · 23.5 KB

ARCHITECTURE.md

File metadata and controls

285 lines (199 loc) · 23.5 KB
Raw

Architecture

How Relune is structured: crate boundaries, data flow, and rules for keeping the CLI and WASM targets aligned.

Table of contents

  1. Goals and constraints
  2. Layers and crates
  3. Request pipeline
  4. Domain model (core)
  5. Dependency rules
  6. Input adapters
  7. Output adapters
  8. Configuration
  9. Diagnostics
  10. Layout
  11. Rendering
  12. WASM boundary
  13. CLI
  14. Security notes
  15. Product evolution
  16. Checklist for new work

1. Goals and constraints

Relune is a reusable schema graph engine with multiple delivery surfaces (CLI, WASM).

Central constraint: domain and pipeline logic must stay target-agnostic. No std::fs in core crates, no wasm-bindgen below the WASM crate, no ad-hoc DB drivers outside introspection. This leads to three design rules:

  1. Explicit intermediate models — schema → graph → layout → render, each testable in isolation
  2. Thin surfaces — CLI and WASM deserialize requests, call relune-app, and serialize results
  3. DTO-style boundaries — public APIs expose Relune-owned types, not parser ASTs or petgraph internals

2. Layers and crates

┌──────────────────────────────────────────────────────────┐
│ Surfaces                                                 │
│   relune-cli              relune-wasm                    │
└────────────────────────────┬─────────────────────────────┘
                             │
                             ▼
┌──────────────────────────────────────────────────────────┐
│ Application                                              │
│   relune-app — validation, config merge, orchestration   │
└────────────────────────────┬─────────────────────────────┘
                             │
         ┌───────────────────┼───────────────────┐
         ▼                   ▼                   ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ Domain / logic   │ │ Input            │ │ Output           │
│ relune-core      │ │ relune-parser-   │ │ relune-render-   │
│ relune-layout    │ │   sql            │ │   theme          │
│                  │ │ relune-introspect│ │ relune-render-   │
│                  │ │                  │ │   svg / html     │
└──────────────────┘ └──────────────────┘ └──────────────────┘
Crate Role
relune-core Normalized schema model, graph construction, filters, lint, diff, review, shared types
relune-layout Hierarchical and force-directed layout, edge routing, overlay annotations, text diagram export (Mermaid, D2, DOT)
relune-parser-sql DDL → Schema (PostgreSQL, MySQL, SQLite; auto-detection)
relune-introspect Live DB metadata → Schema (PostgreSQL, MySQL/MariaDB, SQLite; native builds only)
relune-render-theme Shared theme palette and render-facing theme DTOs used by SVG and HTML renderers
relune-render-svg Layout → SVG string
relune-render-html Layout → self-contained HTML + embedded SVG + viewer scripts
relune-app Use-cases: parse/introspect, render, doc, export, lint, diff, review wiring
relune-cli Args, config TOML, stdin/stdout/files, exit codes
relune-wasm wasm-bindgen façade, JSON in/out
relune-testkit Shared test helpers (tests only)

Repository layout (abbreviated):

crates/
  relune-core/ relune-layout/ relune-parser-sql/ relune-introspect/
  relune-render-theme/ relune-render-svg/ relune-render-html/
  relune-app/ relune-cli/ relune-wasm/ relune-testkit/
fixtures/          # golden inputs and snapshots
docs/              # user-facing guides

3. Request pipeline

Native CLI

SQL file | SQL text | schema JSON | db URL  +  optional relune.toml
    → relune-cli (I/O, load config)
    → relune-app (choose adapter, build pipeline)
    → Schema → graph → layout → (+ optional overlay) → SVG | HTML | Markdown | JSON | diagram text
    → file or stdout

WASM

SQL text | schema JSON | options (from JS)
    → relune-wasm
    → relune-app (same pipeline where applicable)
    → string/JSON result to JS

Introspection and filesystem access stay on the native side; WASM uses in-memory inputs.

4. Domain model (core)

Types live in relune-core (see model.rs, graph.rs, and related modules).

Schema — Top-level container: tables, views, enums. Supports validate() for structural consistency (duplicate names, FK column references, etc.).

Table — TableId, stable_id, optional schema_name, name, columns, foreign_keys, indexes, optional comment.

Column — ColumnId, name, data_type, nullable, is_primary_key, optional comment, optional enum_values for inline enum/set types (e.g. MySQL ENUM('a','b')).

ForeignKey — Optional constraint name, from_columns, to_table, to_columns, on_delete / on_update (ReferentialAction).

View — Parsed and introspected across all three dialects. Stored with the original SQL definition.

Enum — PostgreSQL uses named enum types (CREATE TYPE ... AS ENUM). MySQL has no schema-level enum type; SQL parsing stores inline ENUM(...) / SET(...) definitions on Column.enum_values rather than synthesizing schema-level enum entries (recovered regardless of the resolved dialect, so a misclassified dump keeps its values). Live MySQL introspection currently still lifts inline enum/set column types into Schema.enums. SQLite does not contribute enum metadata.

Identifier normalization — All schema, table, and column identifiers are normalized to lowercase on input (normalize_identifier), and downstream matching (diff, foreign-key resolution, graph construction) is case-insensitive throughout. This is a deliberate simplification, not full SQL quoting semantics: quoted identifiers do not retain their original case, so "User" and "user" collapse to the same name and only one survives. Treat identifier casing as non-significant when feeding schemas into Relune.

Derived artifacts flow through the pipeline:

  • Graph — nodes and edges with stable identities (input to layout)
  • Positioned graph — coordinates and edge paths (output of layout)
  • Render primitives — boxes, paths, labels, grouping (consumed by SVG/HTML renderers)

5. Dependency rules

relune-cli  ──► relune-app ──► relune-core
                  │    ├── relune-layout
                  │    ├── relune-parser-sql
                  │    ├── relune-introspect   (native)
                  │    ├── relune-render-theme
                  │    ├── relune-render-svg
                  │    └── relune-render-html
relune-wasm ───► relune-app
  • relune-core must not depend on CLI, WASM, renderers, or parsers.
  • relune-layout depends on relune-core (not the reverse).
  • relune-render-theme is the shared palette layer for renderers.
  • relune-render-* may depend on relune-core, layout outputs, and relune-render-theme.
  • relune-app composes adapters; avoid duplicating domain rules that belong in core or layout.
  • relune-testkit is for tests; it must not become a default production dependency of shipped crates.

The boundary itself is unchanged, but run_rules now takes an EffectiveDialect argument so dialect-scoped rules dispatch from relune-core while CLI / WASM keep the request DTO surface.

6. Input adapters

Supported paths into a Schema:

Source Crate / module
SQL DDL string or file relune-parser-sql
Normalized schema JSON Deserialized directly into relune-core types
Live database relune-introspect (PostgreSQL, MySQL/MariaDB, SQLite)

relune-app selects the adapter from the request (CLI or WASM DTO). Parsing is pure text; introspection uses read-only metadata queries. PostgreSQL/MySQL/MariaDB introspection applies a default 30 second per-statement deadline, and remote TCP connections default to verifying TLS (PostgreSQL verify-full, MySQL/MariaDB verify-identity) — set sslmode=require / ssl-mode=required in the URL to keep encryption while accepting self-signed clusters. Connection acquisition (including the initial TCP/TLS connect) is bounded by a 30 second acquire timeout, and the entire catalog fetch is bounded by an overall introspection deadline (default 600 seconds, overridable with RELUNE_DB_INTROSPECTION_TIMEOUT_SECS) so backends without an enforceable per-statement deadline — SQLite, or MySQL servers that cannot set a session timeout — still complete in bounded time. Native file-backed SQL and schema JSON inputs are size-limited before reading.

7. Output adapters

Output Producer
Shared palettes / theme DTOs relune-render-theme
SVG relune-render-svg
Self-contained HTML relune-render-html
Markdown documentation relune-app (doc use-case, schema → Markdown)
schema-json / graph-json / layout-json Core + layout serialization
Mermaid erDiagram, D2, Graphviz DOT relune-layout (text from the same positioned graph)

8. Configuration

CLI merges defaults → TOML file → flags for command settings (render, inspect, doc, export, lint, diff). Implementation: crates/relune-cli/src/config.rs. Required inputs still come from the CLI. Named [viewpoints.<name>] presets provide reusable focus/filter/grouping bundles for render and export, and are applied between command defaults and explicit CLI flags. After merge, render/export apply semantic validation for focus depth and filter combinations, and diff file inputs are classified by content instead of extension alone.

9. Diagnostics

Diagnostics are a first-class stream: parse errors, recoverable warnings, unsupported DDL, layout notices, and lint findings. Each carries a stable code and severity suitable for CI (--fail-on-warning, --deny). Partial success (warnings + output) is preferred over hard failure for exploratory use.

LintIssue separates stable identifiers (table_id — matches Table::stable_id) from display names (table_name — human-readable, schema-qualified), and now also carries a review category so CLI / JSON output can group issues into structure, relationships, naming, and documentation review lanes. Renderers and overlay builders use table_id to map issues to diagram nodes without ambiguity in multi-schema environments; CLI and text output use table_name for readability. relune-app::LintResult additionally includes a review summary that records the applied profile, active rule catalog, and suppressed issues after table exceptions.

Migration risk review (relune review) reuses the same diagnostic stream and adds RiskFinding with rule_id (risk/<kebab>), severity (breaking / caution / warning / info), and stable target identifiers (table_id, column_name, fk_name). ReviewRuleId::metadata() / ReviewRuleId::all_metadata() expose a serializable ReviewRuleMetadata { rule_id, default_severity, description } snapshot — the single source of truth for relune review --list-rules, the WASM bindings (returned alongside each response as applied_rule_details), and any downstream rule-legend UI. Per-rule severity overrides from [review.severity_overrides."<rule-id>"] are applied as a post-processing step after rule evaluation and before summary aggregation, so summary counts and the --deny decision both reflect overridden severities.

Lock-risk rules are dialect-scoped: run_rules reads EffectiveDialect and skips a rule whenever its dialect_scope does not include the resolved dialect. risk/add-index-on-large-table, risk/add-fk-on-existing, and risk/alter-column-type activate for Postgres or Mysql; risk/rewrite-table is MySQL-only because the rule encodes MySQL's full-table-rebuild semantics (PostgreSQL handles the same edits without a rewrite). The default profile stays silent in skip cases, but when a caller pins a lock-risk rule via --rules / [review.rules] and the dialect does not match (including Auto, Sqlite, or a Postgres-vs-risk/rewrite-table mismatch), the skip surfaces as a single info-level diagnostic (REVIEW001) so the response explains why no findings were produced.

When the caller leaves --dialect at its default auto, the review use case promotes it to a concrete EffectiveDialect whenever the SQL parser resolved both the before and after inputs to the same dialect (e.g. both look like Postgres → Postgres). DB-URL inputs use the URL scheme as the parser-side signal; schema-JSON inputs carry no parser dialect and stay Auto. If the two sides resolve to different dialects, EffectiveDialect stays Auto and a single warning diagnostic (REVIEW002) is emitted so the user can tell that lock-risk evaluation was skipped because of the mismatch rather than silently. The originally requested dialect and the resolved effective dialect are surfaced in relune-app::ReviewResult (requested_dialect / effective_dialect), in the WASM response, and inline in the CLI text/markdown reports.

10. Layout

relune-layout owns graph layout, overlay annotations, and text diagram exports (Mermaid, D2, DOT). It provides hierarchical and force-directed node placement plus orthogonal backbones for routed edges; renderers can display those routes as orthogonal or curved paths, while straight edges are emitted as direct source-to-target segments. Force-directed mode still uses rank-guided hierarchical seeding to keep requested flow directions stable, then mirrors/swaps the final placement for reversed or horizontal directions without changing the user-facing spacing semantics. Separating it from relune-core keeps a clear boundary between the semantic graph and geometry, and allows targeted benchmarks.

Phases: build layout graph → grouping/focus → layout algorithm → coordinates → auto-tune spacing → global port assignment → obstacle-aware channel selection → parallel edge bundling → self-loop detour handling → label collision avoidance → bounds. Handles cycles, join tables, views, enum references, and multi-schema namespacing.

Routing model — Layout returns one canonical orthogonal backbone for routed edges. Hierarchical routing uses port -> stub -> channel -> stub -> port, then scores inter-rank, same-rank, and reverse-flow channel candidates in a deterministic greedy edge order. Candidate scoring treats obstacle hits, endpoint-side violations, and primary-direction backtracking against rank order as hard constraints before weighted soft costs for clearance, route length, bend count, center deviation, and channel congestion. After route selection, nearby parallel edges on the same channel may share a bundled trunk for readability. orthogonal and curved reuse this backbone, while straight skips control points and renders as a direct source-to-target segment.

Quality passes — After backbone routing, nudge_label shifts edge labels away from overlapping nodes. detour_around_obstacles is no longer part of the non-self-loop path and is only used for self-loop handling, while routing keeps a detour activation count for any non-self-loop edge whose final backbone still intersects padded obstacles. Additionally, auto_tuned adjusts horizontal/vertical spacing based on node count and edge density before coordinate assignment, and port slot offsets keep parallel edges stable on each node side. layout-json exposes this routing state through graph-level and per-edge routing_debug metadata so fixture diffs can explain side policy, slot assignment, and selected channel coordinates directly.

Fixture-level routing regressions are audited in crates/relune-app/tests/fixture_render_audit.rs, which snapshots layout-json and rendered outputs across the main SQL fixtures.

Overlay (overlay module) — A DiagramOverlay attaches annotations (lint warnings, diff status, etc.) to nodes and edges by stable ID, without modifying the positioned graph itself. Renderers accept an optional overlay and apply visual cues (badges, border colors, tooltips) when present. When no overlay is provided the diagram renders normally.

11. Rendering

  • Theme (relune-render-theme) — Shared palettes and theme-facing DTOs consumed by both renderers. ThemeColors includes glow_color and glow_particle fields so hover/highlight effects adapt to light and dark themes.
  • SVG (relune-render-svg) — Geometry, edge paths, labels, themes, optional embedded CSS. Tables, views, and enums share one positioned graph and are styled by node/edge kind. When a DiagramOverlay is provided, the renderer applies severity-colored borders and stroke overrides on affected nodes/edges, adds count badges at the top-right corner of annotated nodes, appends overlay annotation details to <title> tooltips, and adds CSS classes (overlay-error, overlay-warning, etc.) for downstream styling. Visual conventions: header-to-body transition uses a per-node gradient fade; column metadata (PK, FK, IX) is rendered as uniform rounded-rect badges; edge arrow uses an open-chevron marker with userSpaceOnUse for constant size; cardinality markers use enlarged viewBoxes for density resilience.
  • HTML (relune-render-html) — Wraps SVG with interactive behavior (pan/zoom, search, filters, grouping toggles, highlights) and embeds node/edge kind metadata for client-side features. Hover uses a lightweight popover plus subtle 1-hop preview, while click promotes a node into fixed selection and opens the detail drawer. When a DiagramOverlay is provided, annotations are serialized into the issues field of table and edge metadata (JSON), the detail drawer gains a "Health" section listing each issue with severity badge and optional hint, and the object browser displays severity-indicator badges with issue counts on affected tables. Viewer logic is TypeScript under crates/relune-render-html/ts/; bundled JS is committed under crates/relune-render-html/src/js/ and consumed via include_str!. Node + pnpm are required for renderer development when regenerating those bundles, but Rust builds consume the committed assets without installing frontend dependencies at build time.

The two crates are separate to keep low-level vector output apart from document bundling and JS tooling.

12. WASM boundary

  • Export a small, stable API surface (prefer request/response JSON or a few entrypoints).
  • No DB networking or filesystem in the WASM graph path.
  • Deserialize into the same DTOs relune-app uses on native.
  • The public GitHub Pages schema workbench is a thin static client over relune-wasm; it must not fork rendering, inspect, export, lint, diff, or review logic from the CLI path.
  • Review entrypoints are review_from_sql and review_from_schema_json (mirroring diff_from_sql / diff_from_schema_json). Both accept a WasmReviewRequest (deserialized as camelCase from JS) with beforeSql / beforeSchemaJson / afterSql / afterSchemaJson, plus format (text / markdown / json), rules / exceptRules / exceptTables, deny, severityOverrides, and an optional dialect. The response shape is { review, diagnostics, denied, content?, applied_rule_details }: review matches relune-core::ReviewResult for type-safe consumers, content mirrors the CLI’s --format <fmt> string output (the json form matches relune review --format json byte-for-byte so the playground’s download is the same artifact as the CLI), and applied_rule_details carries ReviewRuleMetadata for rule-legend rendering.

13. CLI

relune-cli should stay thin: argument parsing, config load, reading inputs, calling relune-app, writing outputs, mapping errors to exit codes. Parsing, layout, and rendering belong in other crates. --dialect (and the equivalent TOML [review.dialect]) is consumed in two places: the SQL parser uses it to disambiguate dialect-specific syntax, and relune review forwards it as EffectiveDialect so dialect-scoped rule evaluation activates without a separate flag.

The composite GitHub Action under action/ is a thin shell over the same relune-cli binary — it carries no review or domain pipeline logic in YAML or shell. mode: review shells relune review --emit-summary <PATH> once (action/review.sh): the same invocation writes the user-visible report at output-path and a structured JSON summary to a runner-temp path, so has-findings / summary-* can be derived from the summary file independently of the --deny exit code while has-blocking-findings follows the rc directly. This single-pass behavior is purely CI-side orchestration; surface boundaries (CLI, WASM, action) translate requests, while the review pipeline itself stays in relune-app / relune-core.

14. Security notes

  • SQL DDL mode — Parsing only; never executes SQL.
  • Introspection — Read-only metadata queries. The connection URL is fully trusted: Relune connects to exactly the host it names and performs no allow/deny-listing of destinations (no SSRF guard against link-local, private, or cloud-metadata addresses), so never hand it an untrusted DSN. Required privileges: PostgreSQL needs read access to the system catalogs (pg_catalog / information_schema); MySQL/MariaDB needs SELECT on information_schema plus the SHOW VIEW privilege to populate view definitions (without it VIEW_DEFINITION comes back empty, which is normalized to "no definition" and logged); SQLite needs read access to the database file. Servers that cannot set a session statement timeout fall back to the overall introspection deadline.
  • HTML — Self-contained output; escape untrusted names in SVG/HTML layers (maintain parity when adding fields).

15. Product evolution

ERD generator → schema explorer → diff/lint in CI → editor integrations

The explicit intermediate models and crate boundaries exist to support this path without rewriting the core.

16. Checklist for new work

  • Does it keep core logic target-agnostic?
  • Is it deterministically testable (fixtures, snapshots)?
  • Are public types Relune-owned, not leaked third-party internals?
  • If it cannot run on WASM, is it isolated (e.g. behind relune-introspect / CLI only)?
  • Does business logic land in relune-core / relune-layout rather than the CLI?
  • Does it help users understand large schemas (focus, grouping, stable exports), not only “more pixels”?