Skip to content

Add forkable SCI runtime worlds - #1084

Draft
whilo wants to merge 28 commits into
babashka:masterfrom
whilo:forkable-worlds
Draft

Add forkable SCI runtime worlds#1084
whilo wants to merge 28 commits into
babashka:masterfrom
whilo:forkable-worlds

Conversation

@whilo

@whilo whilo commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

This draft adds forkable SCI runtime worlds behind an explicit opt-in:

(sci/init {:runtime-mode :forkable})

The default :standard mode does not allocate a runtime world and retains SCI's direct Var, function, atom, delay, memoize, and namespace paths. In standard mode, sci/fork keeps its historical namespace-environment behavior. Full live-state isolation is enabled only in forkable mode.

This split is intended to make the feature usable by embeddings such as Spindel and Dvergr without imposing its indirections on babashka, nbb, or ordinary SCI users.

Forkable runtime

In forkable mode, sci/fork takes a quiescent snapshot of interpreter-owned state while preserving stable SCI handles. Parent, child, siblings, and repeated descendants then evolve independently.

The implementation uses dense lineage-local slots rather than a persistent-map lookup on every access. It covers:

  • Var roots, metadata, watches, namespaces, and loaded-library state
  • world-scoped dynamic bindings and async world conveyance
  • SCI atoms, volatiles, delays, JVM promises, multimethods, and memoization caches
  • type/protocol state and mutable SCI deftype fields
  • arrays and a host Forkable cooperation protocol
  • rejection of known mutable or affine host resources that cannot be copied safely

The accompanying audit documents the ownership boundary. Arbitrary host closure graphs are not traversed, and external capabilities such as clocks, RNGs, I/O, and running futures require an embedding policy.

Suspended continuations

Forking a quiescent runtime does not implicitly copy a running continuation. Embeddings that suspend interpreted work can now do this explicitly:

(def capsule (sci/capture-continuation-context))
(def child (sci/fork parent))
(def child-capsule
  (sci/retarget-continuation-context capsule child))
(def resume-in-child
  (sci/continuation-context-fn child-capsule continuation))

The capsule is opaque. It captures the complete persistent dynamic-binding frame immediately, preserves binding-box aliases, gives each retarget an independent set of boxes, selects the target world during invocation, unwinds nested bindings correctly, and rejects unrelated SCI lineages. The explicit-context capture arity supports host-invoked interpreted functions after top-level evaluation has returned.

sci/with-detached-context is the complementary host boundary for recursively constructing an independent interpreter without inheriting the caller's active world or binding frame.

Interactive demo

Run a tree of forked REPL worlds with:

clojure -M:examples -m sci.examples.forked-repl

The demo supports :fork NAME, :use ID, and :tree alongside ordinary SCI forms. Each node shows inherited history and later divergence in functions and state created before the fork.

Performance

These are local JVM median-of-medians across three fresh processes. Each process warmed up and took seven samples. The machine was held in its powersave profile, so the relative comparison matters more than the absolute milliseconds.

Probe upstream master :standard :forkable
2M interpreted Var reads 31.3 ms 23.9 ms 59.4 ms
750K interpreted atom derefs 14.1 ms 10.5 ms 34.9 ms
200K interpreted atom swaps 7.8 ms 6.8 ms 26.7 ms
500K interpreted calls 13.2 ms 12.5 ms 32.0 ms

The important result is that standard mode shows no regression in these hot-path probes. Forkable mode still pays a material 2.5-3.9x cost relative to standard mode depending on the operation; this draft exposes that tradeoff rather than making every embedding pay it.

Verification

  • JVM, Clojure 1.10.3: 427 tests, 1671 assertions, 0 failures/errors
  • ClojureDart, Dart 3.12.2: 320 tests, 0 failures/errors
  • Node normal: 454 tests, 6571 assertions, 0 failures/errors
  • Node advanced: 454 tests, 6571 assertions, 0 failures/errors
  • Node no-JIT: 454 tests, 6539 assertions, 0 failures/errors
  • Focused opaque-continuation tests rerun after the final capsule refinement on JVM, ClojureDart, Node normal, Node advanced, and Node no-JIT

This remains a draft for architectural discussion. The commits are intentionally split into reviewable checkpoints and can be decomposed into smaller PRs if that better fits SCI.

Closes #1083

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Explore forkable SCI runtime worlds

1 participant