F# hot reload: Edit-and-Continue delta emission behind --enable:hotreloaddeltas

[NatElkins]

This draft PR presents the complete F# hot reload implementation as a single branch against current main: dotnet watch (with a companion dotnet-watch change, linked below) patches running F# processes in place via the standard EnC pipeline (MetadataUpdater.ApplyUpdate) — the same runtime contract C# uses. Unsupported edits degrade to the rebuild-and-restart flow watch uses today, so a limited scope still behaves as a complete feature.

It is opened as a single draft per discussion with @T-Gro: one branch to review, build, and launch testing from. A decomposition into independently shippable PRs is laid out below and can begin whenever review reaches that stage. Earlier discussion: #11636.

Try it (30–45 min, two clones, copy-paste steps)

docs/hot-reload-quickstart.md walks from git clone to editing a running F# app — including adding a List.map (fun s -> s.ToUpper()) to a live method and watching it apply in place, state preserved. Short version: build this branch, build NatElkins/sdk fsharp-hotreload-watch-v2 (a complete .NET CLI with an F#-aware dotnet watch), copy the freshly built FSharp.Compiler.Service.dll into the SDK layout, add <OtherFlags>$(OtherFlags) --enable:hotreloaddeltas</OtherFlags> to a console app, and dotnet watch run.

What works

• Method body edits, including bodies containing closures, async, resume-point-stable task, and generics
• Lambdas added / edited / removed across generations (new closure classes synthesized into the running process)
• Member additions: methods, module functions and values, static/instance fields, properties, [<CLIEvent>] events
• New type definitions: classes, records, unions, structs, modules, enums, interfaces, delegates, units of measure
• Attribute add/change/remove on existing members; parameter renames
• Multi-project: one watch session, per-project baselines, interleaved edits across an app and its referenced libraries
• Line-shift edits (comment/whitespace above code) become pure sequence-point line updates — no delta, no restart
• Rude edits (same set as C#: signature changes, captured-variable rename/type/scope changes, mid-sequence await insertion, plus F#-specific inline/SRTP) degrade to the standard rebuild-and-restart flow with a precise diagnostic

Isolation and disabling

The feature is designed so that a compile without the flag is indistinguishable from main, and so that the whole feature can be turned off at one point if something goes wrong:

• Off by default. Everything is behind --enable:hotreloaddeltas (requires --debug+, incompatible with --optimize+). No MSBuild property or SDK behavior changes in this repo.
• Flag-off output is byte-identical to main, pinned by the EmittedIL suite (1212 baseline tests) and by dedicated determinism tests (DLL+PDB byte-equality across recompiles and across graph/sequential checking modes).
• Flag-off cost is zero beyond cheap checks. A dedicated audit pass removed all unconditional work from the flag-off path: no reflection, no eager metadata snapshots, no per-name or per-closure side-channel probes, no extended lifetime of the optimized typed tree, and FSharpProjectSnapshot.fs is byte-identical to main (tracked-input staleness lives in the hot reload session layer instead).
• One seam. The compiler driver integration is a single ICompilerEmitHook interface with a no-op default. Guard scripts under tests/scripts/ (run by the verification gate) enforce that only the intended files consume the hook and that the IlxGen name-generation path and fsi surface stay on their pinned shapes.
• Disabling: drop the flag (compiler side); the companion dotnet-watch side additionally has a one-variable kill switch (DOTNET_WATCH_FSHARP_HOTRELOAD=0) that restores stock restart-on-edit behavior.

Architecture

• Sessions are an explicit entity: FSharpChecker.CreateHotReloadSession returns a FSharpHotReloadSession holding per-project committed snapshots + emit baselines — the DebuggingSession/CommittedSolution shape from Roslyn, built on FSharpProjectSnapshot (the snapshot contract, not the experimental workspace surface), so it composes with the FSharpWorkspace direction without depending on it. Solution-wide commit/discard semantics, runtime-capability updates, active-statement intake.
• A typed-tree semantic diff classifies every edit (Roslyn's SemanticEdit/RudeEdit model), gated on the runtime's advertised EnC capabilities (AddMethodToExistingType, NewTypeDefinition, GenericUpdateMethod, …). Anything unclassifiable fails closed.
• Delta emission produces the standard EnC triplet (metadata/IL/PDB deltas with EncLog/EncMap), validated against recorded Roslyn EmitDifference reference deltas, with mdv, and against CoreCLR ApplyUpdate in runtime tests.
• Closure identity is solved the way Roslyn solves it, adapted to F#'s lowering: lambdas get stable identity from a typed-tree occurrence model (ordinal chains + LCS alignment rather than syntax offsets), persisted in Roslyn's exact portable-PDB EnC CDI blob formats (the encoder round-trips Roslyn's own blobs byte-identically), with deterministic occurrence-derived closure-class names so any process can reconstruct identity from the PDB alone.

Design documentation (rendered, on this branch)

Doc	What it covers
hot-reload-architecture.md	Start here. The entity model: FSharpHotReloadSession, per-project committed snapshots + baselines on FSharpProjectSnapshot, the FSharpWorkspace relationship, determinism pins
hot-reload-closure-mapping.md	The closure problem and its solution: lambda occurrence model, Roslyn-format EnC CDI PDB blobs, occurrence-derived deterministic closure naming, cross-process reconstruction; state-machine handling
hot-reload-member-additions.md	Recorded Roslyn EmitDifference reference templates (EncLog/EncMap shapes per edit kind) and the F# emission matrix incl. every intentional fail-closed case
hot-reload-capabilities.md	Runtime capability negotiation (Roslyn EditAndContinueCapabilities parity) and per-capability gating
hot-reload-active-statements.md	The debugger-contract mirror: active statements, sequence-point updates, remapping; host wiring deferred

Scale and review shape

~85 commits, 154 files, ~60k insertions, of which ~34k are tests and ~2.4k docs. The src/ changes are predominantly new self-contained modules (IlxDeltaEmitter.fs, TypedTreeDiff.fs, HotReloadBaseline.fs, the AbstractIL EnC readers, the delta writer stack). Pre-existing files carry hook callouts plus one behavior-neutral refactor (ilwrite.fs MetadataTable record→class with an IEncLogWriter seam); total deletions across the branch are 394 lines.

Proposed path to merging in pieces

The fail-closed design means scope can grow capability by capability — each unsupported case is already a rude edit with a diagnostic, so every intermediate state is a complete, working feature. The natural sequence:

1. Behavior-neutral AbstractIL/ilwrite foundations (no feature, byte-identity evidence) — already staged separately as a 3-commit branch (refactor: AbstractIL EnC foundations (hot reload stack 1/n) NatElkins/fsharp#2)
2. Session entity + typed-tree classification, with every edit classified rude — end-to-end complete at minimal scope (watch restarts on every edit, but through the proper pipeline). This is the PR where the architecture gets decided with reviewers.
3. Method-body-only deltas (the core)
4. Closure mapping + deterministic naming (the one chunk that touches IlxGen/name-generation paths — reviewed on its own)
5. Member/field/type additions, generics, state machines — each already individually capability-gated
6. Active statements / sequence-point updates

Evidence

• ~630 hot-reload tests: typed-tree classification, writer/EncLog shapes asserted against the recorded C# templates, mdv metadata validation, real ApplyUpdate + invocation, multi-generation chains, cross-process (disk-started) sessions
• EmittedIL byte-neutrality: 1212 emitted-IL baseline tests pass with the flag off
• Determinism: baseline capture pins deterministic sequential codegen; DLL+PDB byte-equal across recompiles and across graph/sequential checking modes (pinned by tests). Context: F# compiler produces non-deterministic metadata #Strings heap layout #19732, Determinism: closure type names differ between --parallelcompilation- and --parallelcompilation+ #19928 (open; flag-on compiles are insulated), $(ParallelCompilation) never reaches the Fsc task — targets pass the item @(ParallelCompilation) instead of the property #19935 (why the pin is compiler-level)
• dotnet-watch e2e (companion branch): in-place applies, multi-project interleaved edits, no-op detection, dependency-file handling, rude-edit restart — all green
• Single-command verification gate: ./tests/scripts/hot-reload-verify.sh

Known limitations / future work

• Per-edit apply is ~2–3s; the bulk is an FCS recheck that composes naturally with WIP : Experiment [this repo only] : Fast and unsafe development time compilation out of FSharpChecker caches #19267 (Fast Build from Cache) — the intent is to converge with that work, not duplicate it
• Debugger (VS) host wiring for active statements is deferred; the compiler-side contract mirror is in place
• The remaining intentional rude edits are documented per case in the design docs above

Companion PRs

• refactor: AbstractIL EnC foundations (hot reload stack 1/n) NatElkins/fsharp#2 — stack 1/n: the behavior-neutral AbstractIL/ilwrite foundations
• NatElkins/sdk#1 — the dotnet-watch integration

[WizMe-M]

Hi, I gave a try to hot-reload demo. I followed steps 1-5 and failed. What am I doing wrong?

First of all step 4 says:

Run it:
dotnet watch run --non-interactive
You should see ⌚ F# hot reload session prestarted, then the counter ticking once a second.
But my output was:

Missing `F# hot reload session prestarted`

HotReloadDemo> dotnet watch run --non-interactive
dotnet watch 🔥 Hot reload enabled. For a list of supported edits, see https://aka.ms/dotnet/hot-reload.
dotnet watch 💡 Press Ctrl+R to restart.
Restore complete (0,4s)
    info NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  HotReloadDemo net11.0 succeeded (2,7s) → bin\Debug\net11.0\HotReloadDemo.dll

Build succeeded in 3,7s
dotnet watch ⌚ Loading projects ...
dotnet watch ⌚ Loaded 1 project(s) in 0,4s.
dotnet watch ⌚ Waiting for changes
hello (count: 1) <--------------------------- missing 'F# hot reload session prestarted'

Looks like SDK/F#-compiler with hot-reload wasn't built.

After that I've tried to modify Program.fs and got expected result (TLDR: not works):

TLDR: not works

HotReloadDemo> dotnet watch run --non-interactive
dotnet watch 🔥 Hot reload enabled. For a list of supported edits, see https://aka.ms/dotnet/hot-reload.
dotnet watch 💡 Press Ctrl+R to restart.
Restore complete (0,4s)
    info NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  HotReloadDemo net11.0 succeeded (2,7s) → bin\Debug\net11.0\HotReloadDemo.dll

Build succeeded in 3,7s
dotnet watch ⌚ Loading projects ...
dotnet watch ⌚ Loaded 1 project(s) in 0,4s.
dotnet watch ⌚ Waiting for changes
hello (count: 1)
...
hello (count: 32)
dotnet watch ⌚ File updated: .\Program.fs
hello (count: 33)
...
hello (count: 56)
dotnet watch ⌚ File updated: .\Program.fs
hello (count: 57)
...
hello (count: 68)
dotnet watch 🔄 Restart requested. <------------------- ctrl+R
hello (count: 69)
hello (count: 70)
hello (count: 71)
dotnet watch ⌚ [HotReloadDemo (net11.0)] Exited
dotnet watch 🔄 Restarting.
Restore complete (0,5s)
    info NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  HotReloadDemo net11.0 succeeded (0,2s) → bin\Debug\net11.0\HotReloadDemo.dll

Build succeeded in 1,3s
dotnet watch ⌚ Loading projects ...
dotnet watch ⌚ Loaded 1 project(s) in 0,1s.
dotnet watch ⌚ Waiting for changes
HOT-RELOAD works (count: 1)   <------------------- only after ctrl+R
HOT-RELOAD works (count: 2)

---

Info:

• OS: Windows 11 Pro (ran all steps from Powershell)
• All $env variables were set correcly following manual
• F#-compiler was built with no errors and warnings
• SDK was built on third try with no errors and warnings

[ijklam]

After spending one or two hours debugging this, I found why this cannot work on Windows. The issue is that on the Windows, when the program is running, the program file will be locked. As a result, the "dotnet build" executed by the dotnet-watch's FSharpHotReloadService fails, cause the hot reload cannot work.

I finally have managed to make it work, done some simple test on it, and found some problems:

1. It seems working only when editing the content in the method body.

type Greeter() =
    let mutable count = 0

    member _.Message() =
        count <- count + 1  // Modifying this number is OK
        task {
           do! Task.Delay 2000 // Modifying this number does not take effect until restart
        }
        |> _.GetAwaiter().GetResult()
        sprintf "hello (count: %d)" count  // Modifying this string is OK

let greeter = Greeter()

while true do
    printfn "%s" (greeter.Message())   // Modifying this string doesn't take effect, until restart the program
    System.Threading.Thread.Sleep(1000)   // Changing this 1000 to 2000 doesn't take effect, until restart the program

2. Simply adding a do! ... in a task block will need to restart the program.

    member _.Message() =
        count <- count + 1
        task {
          do! Task.Delay 2000
          do! Task.Delay 2000  // add this line
        }

3. Adding or removing lines around a task needs restart program.

    member _.Message() =
        count <- count + 1  // remove this line
        task {
          do! Task.Delay 2000
        }