From 524d59a0545182f8283236f807dcd76adb3f416c Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Fri, 10 Jul 2026 17:50:48 -0700 Subject: [PATCH 01/10] Add design docs for onnxruntime-web backend consolidation Add two design documents under docs/design/ covering the plan to consolidate onnxruntime-web on the native WebGPU execution provider: - onnxruntime_web_jsep_to_webgpu_ep_migration.md: migrate WebGPU/WebNN from the JSEP TypeScript compute path to the native WebGPU EP, with a transparent default swap, a temporary /jsep escape hatch, int64 analysis, and pre-flip parity investigation items. - onnxruntime_web_remove_webgl_backend.md: deprecate and remove the legacy WebGL (onnxjs) backend, with explicit (non-transparent) removal and current WebGPU browser-coverage context. Both docs use a two-phase (deprecate, then remove) approach, adopt clean removal without tombstone shims, and point consumers at a single canonical tracking issue. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 334 ++++++++++++++++++ .../onnxruntime_web_remove_webgl_backend.md | 202 +++++++++++ 2 files changed, 536 insertions(+) create mode 100644 docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md create mode 100644 docs/design/onnxruntime_web_remove_webgl_backend.md diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md new file mode 100644 index 0000000000000..f3ab677025996 --- /dev/null +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -0,0 +1,334 @@ +# Design: Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP + +**Status:** Draft +**Last updated:** 2026-07-10 +**Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends + +**Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md). +The two efforts are independent but share the `onnxruntime-web/all` bundle and the deprecation-warning utility; see +§5.3 and §8 for the coupling. + +--- + +## 1. Summary + +`onnxruntime-web` implements WebGPU/WebNN compute two ways: + +- **JSEP** — WebGPU (and WebNN) compute implemented in TypeScript over an Asyncify-compiled WASM core. This is + the current default (`.`) and `./all` bundles. +- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM (a port/superset of JSEP). This is + the current `./webgpu` and `./jspi` bundles. + +This document proposes **deprecating and removing JSEP**, standardizing on the **native WebGPU execution +provider** as the single WebGPU/WebNN path. The migration is designed to be **transparent** for the common case +(the default bundle keeps the same `webgpu` backend key and swaps the implementation underneath at build time), +with a **temporary escape hatch** for one release as insurance against undiscovered parity gaps. + +The work is split into two phases: + +- **Phase 1 (this release):** ship deprecation warnings + docs, plumb the one known config gap (`enableInt64`), + and add a temporary `onnxruntime-web/jsep` escape-hatch export. No default behavior change. +- **Phase 2 (next release):** flip the default bundle to the native EP, delete JSEP, and clean up the temporary + build flags and exports. + +--- + +## 2. Motivation + +- **Duplication of maintenance.** JSEP and the native WebGPU EP implement the same operators twice — once in + TypeScript and once in C++. The native EP is the strategic direction and receives the bulk of new op work; + JSEP is effectively a second implementation that must be kept in lockstep. +- **Single, consistent runtime semantics.** Consolidating on the native EP means the browser build shares kernel + behavior with the rest of ONNX Runtime rather than maintaining a parallel TS implementation. +- **Smaller/simpler build matrix.** Removing JSEP lets us delete the temporary `USE_WEBGPU_EP` / `DISABLE_JSEP` + build plumbing. + +--- + +## 3. Goals and non-goals + +### Goals + +- Make the native WebGPU EP the default WebGPU/WebNN backend for `onnxruntime-web`. +- Do so **transparently** for the common consumer (same import, same `webgpu` backend key, same public API). +- Give existing JSEP consumers a clearly-communicated, low-effort migration path plus a one-release safety net. +- Remove the JSEP TypeScript compute path and its build variants. + +### Non-goals + +- **WebGL removal** — tracked separately in + [onnxruntime_web_remove_webgl_backend.md](onnxruntime_web_remove_webgl_backend.md). +- **No change to the Node binding** (`onnxruntime-node` / `ort.node.*`). +- **No change to the React-Native package.** +- **No change to ORT training-web.** +- Not introducing new WebGPU features — this is a consolidation, not a feature effort. + +--- + +## 4. Background: how backend selection works today + +Backend choice is a **build-time** decision, not a runtime toggle. It is gated by `BUILD_DEFS` flags baked into +each bundle at build time: + +- `BUILD_DEFS.DISABLE_JSEP` +- `BUILD_DEFS.DISABLE_WEBGPU` (native EP) +- `BUILD_DEFS.DISABLE_WASM` + +The pivot lives in `js/web/script/build.ts` via a temporary `USE_WEBGPU_EP` flag (default `false`). Its +`DEFAULT_DEFINE` sets `DISABLE_JSEP = !!USE_WEBGPU_EP` and `DISABLE_WEBGPU = !USE_WEBGPU_EP`, so a single flag +chooses JSEP vs. the native EP for a given build. Both `USE_WEBGPU_EP` and `USE_JSPI` are already annotated in +the source as temporary and slated for removal. + +### 4.1 Current bundle / export map (`js/web/package.json`) + +| Export | Artifact | WebGPU/WebNN path | +|---|---|---| +| `.` (default) | `ort.min` / `ort.bundle.min` | JSEP WebGPU + WebNN | +| `./all` | `ort.all.*` | JSEP WebGPU + WebNN (+ WebGL) | +| `./webgpu` | `ort.webgpu.*` | **Native** WebGPU EP + native WebNN | +| `./jspi` | `ort.jspi.*` | Native WebGPU EP + JSPI | +| `./wasm` | `ort.wasm.*` | WASM/CPU only | + +The `.` (default) and `./all` bundles are the **JSEP** WebGPU path today. `./webgpu` and `./jspi` are already the +**native** EP. WebNN in the default/`all` bundles is **JSEP-hosted**; in `./webgpu` it is native. + +The raw WASM artifacts are variant-specific: `ort-wasm-simd-threaded.jsep.wasm` (JSEP), +`ort-wasm-simd-threaded.asyncify.wasm` (native EP, as used by downstream consumers), `*.jspi.wasm`, and the base +`*.wasm`. + +--- + +## 5. Proposed approach + +### 5.1 Transparent default swap (JSEP → native EP) + +The default `.` bundle already registers its WebGPU backend under the `webgpu` registry key. The native EP also +registers under `webgpu`. Because selection is a build-time swap and the registry key is identical, flipping the +default bundle from JSEP to the native EP is **transparent**: existing code that requests the `webgpu` EP +continues to work with no source changes. + +### 5.2 Escape hatch: temporary `onnxruntime-web/jsep` + +Even with strong parity confidence, flipping the default removes the *only* way for a consumer to keep the JSEP +implementation, since: + +- No `/jsep` subpath exists today. +- The only other JSEP-containing bundle is `/all`, which also drags in WebGL. + +To de-risk, Phase 1 ships a **temporary** `onnxruntime-web/jsep` export (built with `USE_WEBGPU_EP=false`) that +pins the JSEP implementation for **one release**. It is marked deprecated and removal-scheduled, and it emits a +one-time warning that doubles as a **bug funnel** — surfacing any native-EP parity gaps before JSEP is deleted. + +### 5.3 `/all` bundle: keep as a converging alias + +`./all` today differs from the default only by including WebGL. After both the WebGL removal (tracked +[separately](onnxruntime_web_remove_webgl_backend.md)) and this JSEP → native swap, its contents converge to +**native WebGPU EP + native WebNN**, i.e. identical to `./webgpu` and the default `.`. + +**Decision:** keep `/all` to avoid breaking existing imports. Implement it as a **real alias to the same physical +artifact** as the webgpu/default bundle rather than a separately-built `ort.all.*` that coincidentally matches — +this avoids bundle drift, drops a build target, and prevents doubling the CDN/WASM payload. The backing WASM also +shifts from `.jsep.wasm` to `.asyncify.wasm` as part of the general WASM-filename migration. The name becomes a +mild misnomer ("all" no longer implies WebGL); this is documented but not worth an export break. + +**Sequencing:** the WebGL-removal effort drops WebGL from `/all` first (while it is still JSEP); this JSEP → native +swap then converges `/all` onto the native artifact. Each effort owns its half of the `/all` transition. + +--- + +## 6. The int64 gap (analysis and decision) + +### 6.1 Mechanics + +The native EP reads int64 support as a **session config entry** keyed +`ep.webgpuexecutionprovider.enableInt64` (constant `kEnableInt64` in +`onnxruntime/core/providers/webgpu/webgpu_provider_options.h`), via `config_options.TryGetConfigEntry` in +`webgpu_provider_factory.cc`. Values are the strings `"1"` / `"0"`. When absent, the C++ default is +`enable_int64 = false`. + +The web layer never sets it through the typed API: `js/web/lib/wasm/session-options.ts` (`case 'webgpu'`) has no +`enableInt64` handling, and there is no field on `WebGpuExecutionProviderOption`. It is, however, reachable today +on the `/webgpu` bundle via the untyped `extra` map, which `iterateExtraOptions` +(`js/web/lib/wasm/wasm-utils.ts`) flattens into dotted session-config keys: + +```js +extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' } +``` + +### 6.2 Emulation semantics (verified) + +Both JSEP and the native EP, **when int64 is enabled on GPU**, use identical **i32-truncated** emulation: + +- Storage type `vec2`, value/compute type `i32`. +- Reads use only the low word (`.x`) and discard the high word (`.y`). +- Writes from `i32` sign-extend into `vec2`. +- All shader arithmetic is 32-bit. + +When int64 is **disabled** (the native default), int64 ops partition to the CPU/WASM fallback using real +`int64_t` — full, correct 64-bit behavior. + +### 6.3 Fidelity vs. the two failure modes + +- **int64 ON (GPU):** maximum JSEP fidelity (byte-identical to JSEP, including its truncation quirks), but lossy + for genuine large-integer int64 *data* (`|v| ≥ 2³¹`): magnitude truncation, 32-bit overflow wrap, and a value + type that literally cannot hold an arbitrary 64-bit result. +- **int64 OFF (CPU):** maximum correctness (true 64-bit), at the cost of a CPU round-trip for those ops. + +In practice the divergence is **usually moot**: int64 in real models carries indices/shapes/axes/counts/token +IDs, all `≪ 2³¹` (a tensor with `> 2³¹` elements cannot fit in a `< 2GB` buffer). So JSEP (GPU-truncated) and +native-int64-OFF (CPU-full) produce **identical numeric results** for realistic models; the difference is +**performance only**. + +**Prior evidence:** `transformers.js` runs the native EP with int64 **off** (Asyncify build, no `extra`) at +scale on int64-heavy token-ID models, confirming the CPU-fallback cost is acceptable in production. + +### 6.4 Decision + +Keep int64 **off by default** everywhere; use a **uniform native-EP config** across `.` and `/webgpu`. int64 is +**not** a correctness blocker for the swap — the risk is performance, not correctness, and migrating JSEP users +(always-truncating) to native-int64-off is a correctness *upgrade*. + +Follow-up actions (not blockers): + +1. Optionally expose a typed `enableInt64?: boolean` on `WebGpuExecutionProviderOption` for discoverability, plus + document the `extra` opt-in. +2. Benchmark an int64-heavy graph (tokenizer / LLM decode) JSEP vs. native-int64-off to quantify CPU-fallback + cost and confirm the perf trade-off. + +### 6.5 Graph-capture interaction (resolved) + +Graph capture and int64 are **coupled inside the EP**, so there is no conflict to design around. In +`webgpu_execution_provider.cc`: + +```cpp +// enable_int64_ is always true when enable_graph_capture_ is true +enable_int64_{config.enable_graph_capture || config.enable_int64}, +``` + +and the graph-capture kernel registry is built with `RegisterKernels(true, true)` (both flags on). Consequences: + +- **Graph capture OFF (default):** int64 stays off → int64 ops on CPU → full 64-bit correctness. +- **Graph capture ON:** the EP **auto-promotes int64 to ON**, keeping int64 ops on-GPU so the captured region is + all-GPU/capturable. + +So leaving `enableInt64` at its default never breaks graph capture — enabling capture flips int64 on +automatically. Graph-capture users implicitly accept the i32-truncation emulation, but that is exactly the +LLM-decode token-ID regime (`≪ 2³¹`) and is already the status quo today. + +--- + +## 7. Open investigation items (to resolve during implementation) + +These are potential parity concerns not yet fully verified in source. The first two could be functional/ +correctness blockers and should be checked before the Phase 2 default flip. + +1. **Proxy-worker parity (potential blocker).** `wasm.proxy = true` spins a dedicated worker + (`js/web/lib/wasm/proxy-wrapper.ts`). JSEP runs WebGPU compute over Asyncify; the native EP has its own + threading assumptions. `transformers.js` forces `proxy = false`, so it is **not** prior-art for the proxied + path. Confirm the native EP behaves identically under `wasm.proxy = true`. +2. **IO-binding parity (potential blocker).** Both `'gpu-buffer'` and `'ml-tensor'` output locations are wired in + `js/web/lib/wasm/wasm-core-impl.ts`. Verify the native EP provides the same zero-copy GPU-buffer / ML-tensor + handoff semantics. This will not be caught by op-parity tests. +3. **WebNN native-vs-JSEP parity.** In the default `.` bundle, WebNN is JSEP-hosted; the flip moves those users + to native WebNN as well — a second silent backend swap under the same default bundle. Validate separately from + WebGPU. +4. **Typed-option parity.** Audit whether any typed `WebGpuExecutionProviderOption` field is honored by only one + backend (cache modes, `preferredLayout`, buffer-pool knobs, `forceCpuNodeNames`, etc.). A JSEP-only field + silently becomes a no-op under the native EP. + +--- + +## 8. Phase 1 — Deprecation (this release) + +No default behavior change. Deliverables: + +1. **JSEP warn-once.** In `wasm-core-impl.ts` `initEp` inside the `if (!BUILD_DEFS.DISABLE_JSEP)` block, for + `epName` `'webgpu'` (and `'webnn'`). Only JSEP builds warn; native-EP builds never warn. +2. **Shared deprecation-warning utility.** Once-only, respects `env.logLevel`, with an opt-out consideration. + Shared with the WebGL-removal effort. +3. **int64 plumbing (prereq).** Plumb `enableInt64` into `js/web/lib/wasm/session-options.ts` (mirroring the + `enableGraphCapture` handling) so migrating users can opt in if they need JSEP-identical GPU int64. +4. **Escape hatch.** Add the temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`), marked + deprecated with a scheduled removal release. +5. **Docs.** Deprecation banners + migration guidance in `js/web/README` and `js/web/docs/webgpu-operators.md`; + release notes / CHANGELOG entries. + +### Warning placement rationale + +- **Default `.` bundle (still JSEP in Phase 1):** the JSEP warn-once (#1) applies. After the Phase 2 flip it + becomes native EP and emits nothing — the swap is covered in release notes, not a runtime warning, because it + is not actionable for the consumer. +- **`/jsep` escape-hatch bundle:** warns once ("deprecated JSEP build, removed in vX; file an issue if the native + WebGPU EP does not work for you"). Doubles as a parity bug funnel. + +--- + +## 9. Phase 2 — Removal (next release) + +1. **Flip default:** native WebGPU EP becomes the `webgpu` backend in `ort` / `ort.all`; remove the temporary + `USE_WEBGPU_EP` flag. +2. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports. Repoint + `/all` to the webgpu/default artifact (§5.3). +3. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. +4. **Relocate WebNN** out of the `jsep/` directory (`backend-webnn.ts`, `webnn/`) to a neutral path. +5. **Remove `pre-jsep.js` glue;** confirm `post-webgpu.js` / `post-webnn.js` cover all initialization. +6. **Remove the temporary `onnxruntime-web/jsep` export.** + +### Removal ergonomics (no tombstones) + +The `/jsep` export is removed **cleanly** — no throwing-stub "tombstone" module and no runtime shim are left +behind. A consumer still importing `onnxruntime-web/jsep` after removal gets the native bundler error ("subpath +./jsep is not defined by exports"), which is an acceptable, immediate build-time failure on this temporary, +one-release, opt-in surface. The default `.` import is unaffected (it silently swaps to the native EP), so no +consumer following the documented path hits an error. The removal is communicated via release notes and the +pinned tracking issue rather than a runtime shim. + +### Release gate + +Before flipping the default, the **web CI test matrix must run the default-bundle suite against the native EP**. +Today that suite exercises JSEP for the `.` bundle; a green `/webgpu` run is not sufficient to prove the default +flip. This is an explicit gate on Phase 2. + +--- + +## 10. Risks and mitigations + +| Risk | Likelihood | Mitigation | +|---|---|---| +| Undiscovered native-EP parity gap vs. JSEP | Medium | Temporary `/jsep` escape hatch (1 release) + warn-once bug funnel; differential tests | +| int64 perf regression (CPU fallback) | Low–Med | int64-off is correctness-preserving; benchmark int64-heavy graph; document `extra`/typed opt-in | +| Proxy-worker path behaves differently | Unknown | **Investigate before flip** (§7.1) | +| IO-binding (gpu-buffer/ml-tensor) semantics differ | Unknown | **Investigate before flip** (§7.2) | +| WebNN behavior changes under native path | Unknown | Validate native WebNN separately (§7.3) | +| WASM artifact filename change breaks `wasmPaths` | Medium | Document migration; call out `.jsep.wasm` → `.asyncify.wasm` | +| No telemetry on JSEP adoption | Medium | Warn-once funnel + one-release escape hatch as the feedback mechanism | + +--- + +## 11. Migration guide (for consumers) + +- **Default import (`onnxruntime-web`):** no code change needed. The `webgpu` backend continues to work; the + implementation swaps to the native EP in the removal release. +- **`onnxruntime-web/all`:** continues to work and converges to the native EP. +- **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). Please file an issue + if the native WebGPU EP does not work for your model so the gap can be fixed before JSEP is deleted. +- **int64-heavy models needing JSEP-identical GPU behavior:** set + `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }` (or the typed option once available). Note this is + lossy for genuine large-integer int64 data; the default (CPU fallback) is more correct. +- **`wasmPaths`:** the backing WASM artifact changes (`.jsep.wasm` → `.asyncify.wasm`); update any hardcoded + paths. + +> **Canonical source:** the authoritative, continuously-updated migration guidance for both the JSEP and WebGL +> removals lives in a single pinned tracking issue (link TBD). Console warnings, README/docs, and CHANGELOG +> entries **link** to that issue rather than duplicating this guidance, so there is one place to keep current. + +--- + +## 12. Open questions + +1. **Default-flip timing.** Phase 1 vs. Phase 2. Recommendation: Phase 2 (conservative). +2. **Warning suppressibility.** Always-once vs. env opt-out. Recommendation: once + respect `logLevel` so + error/fatal silences it. +3. **WebNN-on-JSEP messaging in Phase 1.** Recommendation: yes — note that `ort` / `ort.all` WebNN moves to the + native path. +4. **int64 typed option in Phase 1.** Recommendation: yes (discoverability), default off. diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md new file mode 100644 index 0000000000000..c17014b422660 --- /dev/null +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -0,0 +1,202 @@ +# Design: Remove the WebGL (onnxjs) backend from onnxruntime-web + +**Status:** Draft +**Last updated:** 2026-07-10 +**Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGL backend + +**Related work:** +[Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP](onnxruntime_web_jsep_to_webgpu_ep_migration.md). +The two efforts are independent but share the `onnxruntime-web/all` bundle and the deprecation-warning utility; +see §5 and §6 for the coupling. + +--- + +## 1. Summary + +`onnxruntime-web` ships a legacy **WebGL** backend — the `onnxjs` implementation under `js/web/lib/onnxjs`, +registered under the `'webgl'` backend key. It predates the current WASM architecture, supports a narrower +operator set, and has known fp32/behavioral drift relative to the WebGPU path. + +This document proposes **deprecating and then removing** the WebGL backend. Unlike the JSEP → native WebGPU EP +migration, this is a straightforward **deprecate-and-delete** with an explicit migration message — there is no +transparent redirect (see §5.1). + +The work is split into two phases: + +- **Phase 1 (this release):** ship a deprecation warning + docs. No behavior change. +- **Phase 2 (next release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` build + variant. + +--- + +## 2. Motivation + +- **Legacy and unmaintained.** The `onnxjs` WebGL backend predates the WASM/EP architecture and does not receive + new operator or feature work. +- **Narrower op coverage and behavioral drift.** WebGL supports fewer operators than WebGPU and exhibits fp32/op + differences, making it an inconsistent fallback. +- **Build-matrix simplification.** Removing WebGL deletes the `ort.webgl` bundle variant and the + `BUILD_DEFS.DISABLE_WEBGL` plumbing, and drops WebGL from the `ort.all` bundle. + +--- + +## 3. Goals and non-goals + +### Goals + +- Remove the WebGL (`onnxjs`) backend from `onnxruntime-web`. +- Give existing WebGL consumers a clear, actionable migration path (to the WebGPU EP or the WASM/CPU backend). +- Remove the `ort.webgl` build variant and drop WebGL from `ort.all`. + +### Non-goals + +- **JSEP → native WebGPU EP migration** — tracked separately in + [onnxruntime_web_jsep_to_webgpu_ep_migration.md](onnxruntime_web_jsep_to_webgpu_ep_migration.md). +- **No change to the Node binding, React-Native package, or ORT training-web.** + +--- + +## 4. Background + +The WebGL backend is the `onnxjs` implementation in `js/web/lib/onnxjs`, exposed via `backend-onnxjs.ts` and +registered under the `'webgl'` backend key with **negative priority** — meaning it is excluded from the default +fallback ordering and only used when explicitly requested (`executionProviders: ['webgl']`). + +It appears in two bundles: + +| Export | Artifact | Contents | +|---|---|---| +| `./webgl` | `ort.webgl.*` | WebGL only (`DISABLE_WASM: true`) | +| `./all` | `ort.all.*` | JSEP WebGPU + WebNN **+ WebGL** | + +The `./webgl` bundle sets `BUILD_DEFS.DISABLE_WASM: true`, so it has **no WASM/CPU fallback** — it is WebGL or +nothing. + +### 4.1 WebGPU browser coverage (why the fallback gap is small and shrinking) + +A common historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that +is **no longer true**. Per MDN's `api.GPU` browser-compat data: + +| Browser | WebGPU | +|---|---| +| Chrome / Edge (desktop) | ✔️ 113+ | +| Chrome Android | ✔️ 121+ | +| Safari (macOS) | ✔️ 26 | +| Safari (iOS) | ✔️ 26 | +| Chrome / Edge (iOS, via WebKit) | ✔️ 26 | +| Firefox (desktop) | ✔️ 141 | +| Firefox for Android | ❌ | + +WebGPU is now broadly available across current Chrome/Edge, Safari 26 (macOS **and** iOS), and Firefox 141. The +remaining gap is older browser versions, Firefox for Android, and some Linux configurations. This narrows the set +of users for whom WebGL is the *only* GPU path to a small, shrinking tail and strengthens the case for a clean +removal. + +--- + +## 5. Proposed approach + +### 5.1 Explicit removal, not a transparent redirect + +Unlike the WebGPU JSEP → native swap, WebGL cannot be transparently redirected to another backend: + +- The `./webgl` bundle sets `DISABLE_WASM: true`, so there is **no WASM/CPU fallback** to silently fall back to. +- WebGPU requires `navigator.gpu`; a silent redirect would fail on devices without WebGPU support. +- WebGL exhibits fp32/op drift, so silently swapping results would be a hidden behavioral change. + +Therefore WebGL gets an **explicit deprecation warning + removal** with an actionable migration message directing +users to the WebGPU EP (`webgpu`) or the WASM/CPU backend (`wasm`). + +### 5.2 Warning placement + +The warning fires in `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), in `init()` / +`createInferenceSessionHandler`. Because the backend registers with negative priority, this only fires when WebGL +is **explicitly requested**, so it does not spam consumers who never opt into WebGL. It fires for both `ort.all` +and `ort.webgl`. + +### 5.3 Removal ergonomics (no tombstones) + +When the backend is removed in Phase 2, we do **not** ship throwing-stub "tombstone" modules for the `./webgl` +export or a runtime shim for the `'webgl'` backend key. The rationale: + +- Removing the `./webgl` export means a lingering `import 'onnxruntime-web/webgl'` fails with the native bundler + error ("subpath ./webgl is not defined by exports"). That is a clear, immediate, build-time failure — no shim + needed. +- Requesting `executionProviders: ['webgl']` alone after removal already throws the generic "no available backend + found" error from `resolveBackendAndExecutionProviders`. +- WebGL was never a production-grade, first-class backend (narrow op set, fp32 drift, negative priority), so an + elaborate soft-landing is not warranted. + +The **one** silent case worth guarding is `executionProviders: ['webgl', 'wasm']`: today the unavailable +`'webgl'` entry is silently dropped and the session runs on `wasm`. This is a *correct* outcome (the consumer +gets a working fallback), but it is silent. An optional, cheap safeguard is a **single non-silent** warn in +`resolveBackendAndExecutionProviders` when a removed backend name is dropped, pointing at the migration guide. +Given the now-broad WebGPU coverage (§4.1) and the fact that `wasm` still runs the model, this is a low-priority +nicety, not a blocker — Phase 1's explicit deprecation warning already does the real communication. + +--- + +## 6. `/all` bundle coupling + +WebGL is one of the two backends in `./all` (the other being JSEP WebGPU/WebNN). Removing WebGL drops it from +`/all`, leaving JSEP (which the +[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) subsequently converges to the native +EP). This effort owns **removing WebGL from `/all`**; the JSEP effort owns the **final convergence** of `/all` +onto the native artifact. `/all` itself is **kept** as an export to avoid breaking imports (see the JSEP doc §5.3 +for the alias decision). + +--- + +## 7. Phase 1 — Deprecation (this release) + +No behavior change. Deliverables: + +1. **WebGL warn-once.** In `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), `init()` / + `createInferenceSessionHandler`. Gate on `logLevel`; dedupe via a module flag. Uses the shared + deprecation-warning utility (built by the JSEP effort, or here — whichever lands first). +2. **Docs.** Deprecation banner + migration guidance in `js/web/README` and `js/web/docs/webgl-operators.md`; + release notes / CHANGELOG entries. + +--- + +## 8. Phase 2 — Removal (next release) + +1. **Delete WebGL:** remove the `onnxjs` backend (`js/web/lib/onnxjs`), the `'webgl'` registration, and + `backend-onnxjs.ts`. +2. **Remove the `ort.webgl` build variant;** update `build.ts` and `package.json` exports (remove the `./webgl` + export). +3. **Drop WebGL from `ort.all`** (see §6). +4. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_WEBGL` and the code it gates; simplify `index.ts`. + +--- + +## 9. Risks and mitigations + +| Risk | Likelihood | Mitigation | +|---|---|---| +| Consumers relying on WebGL as a fallback where WebGPU is unavailable | Low–Medium | WebGPU now broadly available (§4.1), shrinking the affected tail; explicit migration message to `wasm`; deprecation window + release notes | +| Silent breakage from `./webgl` import removal | Low | Deprecate first; document removal release; no transparent redirect (avoids hidden drift) | +| No telemetry on WebGL adoption | Medium | Warn-once as the feedback mechanism during the deprecation window | + +--- + +## 10. Migration guide (for consumers) + +- **`onnxruntime-web/webgl`:** removed. Migrate to the WebGPU EP (`executionProviders: ['webgpu']`) or the + WASM/CPU backend (`executionProviders: ['wasm']`). There is no automatic redirect because WebGL builds have no + WASM fallback and WebGPU requires `navigator.gpu`. +- **`onnxruntime-web/all`:** continues to work, but no longer includes WebGL. If you relied on WebGL as a + fallback via `/all`, add `wasm` to your `executionProviders` list. + +> **Canonical source:** the authoritative, continuously-updated migration guidance for both the WebGL and JSEP +> removals lives in a single pinned tracking issue (link TBD). Console warnings, README/docs, and CHANGELOG +> entries **link** to that issue rather than duplicating this guidance, so there is one place to keep current. + +--- + +## 11. Open questions + +1. **Warning suppressibility.** Always-once vs. env opt-out. Recommendation: once + respect `logLevel` so + error/fatal silences it. (Consistent with the JSEP effort.) +2. **Deprecation window length.** One release (aligned with the JSEP removal) vs. longer. Recommendation: align + with the JSEP removal for a single coordinated backend-consolidation message. From fc62ff6bda717d01ae133a1b976018a9805d266a Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Fri, 10 Jul 2026 18:06:06 -0700 Subject: [PATCH 02/10] Address design-doc review feedback Revise the onnxruntime-web backend consolidation design docs per review comments: - WebGL removal doc: reword the WebGPU browser-coverage section to reflect MDN's partial support with platform caveats (Chrome Linux GPU-gen limits; Firefox 141 excluding Linux and Intel Macs) instead of implying universal availability, and revert the fallback-risk likelihood to Medium. - WebGL removal doc: drop the paragraph about the ['webgl','wasm'] case, since the existing resolveBackendAndExecutionProviders warning already covers it and no special handling is needed. - JSEP migration doc: tighten the Phase 2 release gate to explicitly require resolving the Section 7 blockers (proxy-worker, gpu-buffer/ml-tensor IO binding, native WebNN) with targeted coverage, in addition to running the default-bundle suite against the native EP. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 24 ++++++++++-- .../onnxruntime_web_remove_webgl_backend.md | 38 +++++++++---------- 2 files changed, 38 insertions(+), 24 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index f3ab677025996..5df73be08b45b 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -285,9 +285,27 @@ pinned tracking issue rather than a runtime shim. ### Release gate -Before flipping the default, the **web CI test matrix must run the default-bundle suite against the native EP**. -Today that suite exercises JSEP for the `.` bundle; a green `/webgpu` run is not sufficient to prove the default -flip. This is an explicit gate on Phase 2. +Phase 2 (the default flip) is gated on **both** of the following: + +1. **Default-bundle suite runs against the native EP.** The web CI test matrix must exercise the default `.` + bundle against the native WebGPU EP. Today that suite runs JSEP for the `.` bundle; a green `/webgpu` run is + **not** sufficient to prove the default flip, because the two bundles differ in wiring (proxy, IO-binding, + WebNN host) even when op kernels match. +2. **Section 7 blockers resolved with targeted coverage.** The potential blockers in §7 must be closed out before + the flip — not merely tracked. Op-parity tests do **not** exercise these paths, so each needs dedicated + coverage: + - **Proxy-worker path (§7.1):** run the native EP under `wasm.proxy = true` in CI (not just the default + `proxy = false`). + - **IO-binding (§7.2):** add tests for both `'gpu-buffer'` and `'ml-tensor'` output locations that assert the + native EP's zero-copy handoff semantics. + - **Native WebNN (§7.3):** validate the native WebNN path separately from WebGPU, since the default-bundle + WebNN users are silently moved from JSEP-hosted to native WebNN by the same flip. + + §7.4 (typed-option parity) should be audited in the same pass; a JSEP-only option silently becoming a no-op is + a quieter regression than the first three but is closed the same way (test or documented removal). + +Passing gate #1 while leaving gate #2 open would let the design meet its stated bar with acknowledged +blocker-class parity gaps still unverified; both are required. --- diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index c17014b422660..a769bede7b409 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -72,25 +72,28 @@ It appears in two bundles: The `./webgl` bundle sets `BUILD_DEFS.DISABLE_WASM: true`, so it has **no WASM/CPU fallback** — it is WebGL or nothing. -### 4.1 WebGPU browser coverage (why the fallback gap is small and shrinking) +### 4.1 WebGPU browser coverage (expanding, but with platform caveats) A common historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that -is **no longer true**. Per MDN's `api.GPU` browser-compat data: +is **less true than it was**, though MDN's `api.GPU` browser-compat data still reports meaningful caveats — the +Chrome and Firefox entries are **partial**, not blanket support: -| Browser | WebGPU | +| Browser | WebGPU (per MDN BCD) | |---|---| -| Chrome / Edge (desktop) | ✔️ 113+ | -| Chrome Android | ✔️ 121+ | -| Safari (macOS) | ✔️ 26 | -| Safari (iOS) | ✔️ 26 | -| Chrome / Edge (iOS, via WebKit) | ✔️ 26 | -| Firefox (desktop) | ✔️ 141 | +| Chrome / Edge (desktop) | Partial from 113 (ChromeOS/macOS/Windows); broader/full only in a later version, with Linux limited to newer Intel-gen GPUs | +| Chrome Android | 121+ | +| Safari (macOS) | Full, 26 | +| Safari (iOS) | Full, 26 | +| Chrome / Edge (iOS, via WebKit) | Full, 26 | +| Firefox (desktop) | Partial from 141; **excludes Linux and Intel-based macOS** (Apple-Silicon-first) | | Firefox for Android | ❌ | -WebGPU is now broadly available across current Chrome/Edge, Safari 26 (macOS **and** iOS), and Firefox 141. The -remaining gap is older browser versions, Firefox for Android, and some Linux configurations. This narrows the set -of users for whom WebGL is the *only* GPU path to a small, shrinking tail and strengthens the case for a clean -removal. +So Safari/iOS coverage is now solid, and Chrome/Firefox are moving in the right direction — but there remain real +gaps: older browser versions, Firefox for Android, Linux (both Chrome's GPU-generation limits and Firefox's +exclusion), and Intel-based Macs on Firefox. WebGPU coverage is **broadening but not yet universal**, so the set +of users for whom WebGL is the *only* GPU path is shrinking rather than gone. This supports removing WebGL over a +deprecation window, but the design does **not** rely on WebGPU being universally available — the actionable +fallback for uncovered users is the WASM/CPU backend, not WebGPU. --- @@ -127,13 +130,6 @@ export or a runtime shim for the `'webgl'` backend key. The rationale: - WebGL was never a production-grade, first-class backend (narrow op set, fp32 drift, negative priority), so an elaborate soft-landing is not warranted. -The **one** silent case worth guarding is `executionProviders: ['webgl', 'wasm']`: today the unavailable -`'webgl'` entry is silently dropped and the session runs on `wasm`. This is a *correct* outcome (the consumer -gets a working fallback), but it is silent. An optional, cheap safeguard is a **single non-silent** warn in -`resolveBackendAndExecutionProviders` when a removed backend name is dropped, pointing at the migration guide. -Given the now-broad WebGPU coverage (§4.1) and the fact that `wasm` still runs the model, this is a low-priority -nicety, not a blocker — Phase 1's explicit deprecation warning already does the real communication. - --- ## 6. `/all` bundle coupling @@ -174,7 +170,7 @@ No behavior change. Deliverables: | Risk | Likelihood | Mitigation | |---|---|---| -| Consumers relying on WebGL as a fallback where WebGPU is unavailable | Low–Medium | WebGPU now broadly available (§4.1), shrinking the affected tail; explicit migration message to `wasm`; deprecation window + release notes | +| Consumers relying on WebGL as a fallback where WebGPU is unavailable | Medium | WebGPU coverage is broadening but still has platform caveats (§4.1); the actionable fallback for uncovered users is `wasm`, not WebGPU; explicit migration message + deprecation window + release notes | | Silent breakage from `./webgl` import removal | Low | Deprecate first; document removal release; no transparent redirect (avoids hidden drift) | | No telemetry on WebGL adoption | Medium | Warn-once as the feedback mechanism during the deprecation window | From 288864a74e02206922291c0b8ba1669222d2ebdb Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Tue, 14 Jul 2026 11:55:37 -0700 Subject: [PATCH 03/10] Tighten onnxruntime-web backend deprecation design docs Condense both the JSEP -> native WebGPU EP migration and the WebGL removal design docs, trimming the derivation narrative while preserving the settled decisions and rationale. Collapse the int64 analysis subsections into a single section, reduce the open-investigation source/runtime status to concise lines, and reframe the removal-ergonomics sections around what each phase does rather than what it omits. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 337 ++++++++---------- .../onnxruntime_web_remove_webgl_backend.md | 101 +++--- 2 files changed, 188 insertions(+), 250 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index 5df73be08b45b..ccbf071c3600a 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -1,12 +1,11 @@ # Design: Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP **Status:** Draft -**Last updated:** 2026-07-10 +**Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends -**Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md). -The two efforts are independent but share the `onnxruntime-web/all` bundle and the deprecation-warning utility; see -§5.3 and §8 for the coupling. +**Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md) +— independent, but shares the `onnxruntime-web/all` bundle and the deprecation-warning utility (see §5.3, §8). --- @@ -14,22 +13,21 @@ The two efforts are independent but share the `onnxruntime-web/all` bundle and t `onnxruntime-web` implements WebGPU/WebNN compute two ways: -- **JSEP** — WebGPU (and WebNN) compute implemented in TypeScript over an Asyncify-compiled WASM core. This is - the current default (`.`) and `./all` bundles. -- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM (a port/superset of JSEP). This is - the current `./webgpu` and `./jspi` bundles. +- **JSEP** — WebGPU (and WebNN) compute implemented in TypeScript over an Asyncify-compiled WASM core. Powers the + current default (`.`) and `./all` bundles. +- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM (a port/superset of JSEP). Powers the + current `./webgpu` and `./jspi` bundles. -This document proposes **deprecating and removing JSEP**, standardizing on the **native WebGPU execution -provider** as the single WebGPU/WebNN path. The migration is designed to be **transparent** for the common case -(the default bundle keeps the same `webgpu` backend key and swaps the implementation underneath at build time), -with a **temporary escape hatch** for one release as insurance against undiscovered parity gaps. +This document proposes **deprecating and removing JSEP**, standardizing on the **native WebGPU EP** as the single +WebGPU/WebNN path. The swap is **transparent** for the common case — the default bundle keeps the `webgpu` backend +key and changes implementation at build time — with a **temporary one-release escape hatch** as insurance against +undiscovered parity gaps. -The work is split into two phases: - -- **Phase 1 (this release):** ship deprecation warnings + docs, plumb the one known config gap (`enableInt64`), - and add a temporary `onnxruntime-web/jsep` escape-hatch export. No default behavior change. -- **Phase 2 (next release):** flip the default bundle to the native EP, delete JSEP, and clean up the temporary - build flags and exports. +- **Phase 1 (this release):** flip the default (`.`) and `./all` bundles to the native EP, add the temporary + `onnxruntime-web/jsep` escape-hatch export, and ship deprecation warnings + docs. The flip and the hatch ship + together, so the native default and its JSEP fallback coexist for exactly one release. The pinned tracking issue + is published **ahead of** this release as the early-warning channel — no separate warning-only release (§8). +- **Phase 2 (next release):** delete JSEP and remove the temporary `/jsep` export and build flags. --- @@ -109,203 +107,153 @@ continues to work with no source changes. ### 5.2 Escape hatch: temporary `onnxruntime-web/jsep` -Even with strong parity confidence, flipping the default removes the *only* way for a consumer to keep the JSEP -implementation, since: - -- No `/jsep` subpath exists today. -- The only other JSEP-containing bundle is `/all`, which also drags in WebGL. - -To de-risk, Phase 1 ships a **temporary** `onnxruntime-web/jsep` export (built with `USE_WEBGPU_EP=false`) that -pins the JSEP implementation for **one release**. It is marked deprecated and removal-scheduled, and it emits a -one-time warning that doubles as a **bug funnel** — surfacing any native-EP parity gaps before JSEP is deleted. +After the flip, no existing bundle keeps JSEP (no `/jsep` subpath exists today, and the flip moves both `.` and +`/all` to the native EP). To de-risk, Phase 1 adds a **temporary** `onnxruntime-web/jsep` export (built +`USE_WEBGPU_EP=false`) that pins JSEP for **one release**. It is marked deprecated with a scheduled removal and +emits a one-time warning that doubles as a **parity bug funnel** — surfacing native-EP gaps before JSEP is +deleted. ### 5.3 `/all` bundle: keep as a converging alias -`./all` today differs from the default only by including WebGL. After both the WebGL removal (tracked -[separately](onnxruntime_web_remove_webgl_backend.md)) and this JSEP → native swap, its contents converge to -**native WebGPU EP + native WebNN**, i.e. identical to `./webgpu` and the default `.`. +`./all` today differs from the default only by including WebGL. After the WebGL removal (tracked +[separately](onnxruntime_web_remove_webgl_backend.md)) and this JSEP → native swap, it converges to **native +WebGPU EP + native WebNN**, identical to `./webgpu` and the default `.`. -**Decision:** keep `/all` to avoid breaking existing imports. Implement it as a **real alias to the same physical -artifact** as the webgpu/default bundle rather than a separately-built `ort.all.*` that coincidentally matches — -this avoids bundle drift, drops a build target, and prevents doubling the CDN/WASM payload. The backing WASM also -shifts from `.jsep.wasm` to `.asyncify.wasm` as part of the general WASM-filename migration. The name becomes a -mild misnomer ("all" no longer implies WebGL); this is documented but not worth an export break. +**Decision:** keep `/all` to avoid breaking imports, implemented as a **real alias to the same physical artifact** +as the webgpu/default bundle rather than a separately-built `ort.all.*` — avoiding bundle drift, dropping a build +target, and not doubling the CDN/WASM payload. The backing WASM shifts `.jsep.wasm` → `.asyncify.wasm` with the +general filename migration. "all" becomes a mild misnomer (no longer implies WebGL) — documented, not worth an +export break. -**Sequencing:** the WebGL-removal effort drops WebGL from `/all` first (while it is still JSEP); this JSEP → native -swap then converges `/all` onto the native artifact. Each effort owns its half of the `/all` transition. +**Sequencing:** the WebGL effort drops WebGL from `/all` first (still JSEP); this swap then converges `/all` onto +the native artifact. Each effort owns its half. --- ## 6. The int64 gap (analysis and decision) -### 6.1 Mechanics - -The native EP reads int64 support as a **session config entry** keyed -`ep.webgpuexecutionprovider.enableInt64` (constant `kEnableInt64` in -`onnxruntime/core/providers/webgpu/webgpu_provider_options.h`), via `config_options.TryGetConfigEntry` in -`webgpu_provider_factory.cc`. Values are the strings `"1"` / `"0"`. When absent, the C++ default is -`enable_int64 = false`. - -The web layer never sets it through the typed API: `js/web/lib/wasm/session-options.ts` (`case 'webgpu'`) has no -`enableInt64` handling, and there is no field on `WebGpuExecutionProviderOption`. It is, however, reachable today -on the `/webgpu` bundle via the untyped `extra` map, which `iterateExtraOptions` -(`js/web/lib/wasm/wasm-utils.ts`) flattens into dotted session-config keys: - -```js -extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' } -``` - -### 6.2 Emulation semantics (verified) - -Both JSEP and the native EP, **when int64 is enabled on GPU**, use identical **i32-truncated** emulation: - -- Storage type `vec2`, value/compute type `i32`. -- Reads use only the low word (`.x`) and discard the high word (`.y`). -- Writes from `i32` sign-extend into `vec2`. -- All shader arithmetic is 32-bit. - -When int64 is **disabled** (the native default), int64 ops partition to the CPU/WASM fallback using real -`int64_t` — full, correct 64-bit behavior. - -### 6.3 Fidelity vs. the two failure modes - -- **int64 ON (GPU):** maximum JSEP fidelity (byte-identical to JSEP, including its truncation quirks), but lossy - for genuine large-integer int64 *data* (`|v| ≥ 2³¹`): magnitude truncation, 32-bit overflow wrap, and a value - type that literally cannot hold an arbitrary 64-bit result. -- **int64 OFF (CPU):** maximum correctness (true 64-bit), at the cost of a CPU round-trip for those ops. - -In practice the divergence is **usually moot**: int64 in real models carries indices/shapes/axes/counts/token -IDs, all `≪ 2³¹` (a tensor with `> 2³¹` elements cannot fit in a `< 2GB` buffer). So JSEP (GPU-truncated) and -native-int64-OFF (CPU-full) produce **identical numeric results** for realistic models; the difference is -**performance only**. - -**Prior evidence:** `transformers.js` runs the native EP with int64 **off** (Asyncify build, no `extra`) at -scale on int64-heavy token-ID models, confirming the CPU-fallback cost is acceptable in production. - -### 6.4 Decision - -Keep int64 **off by default** everywhere; use a **uniform native-EP config** across `.` and `/webgpu`. int64 is -**not** a correctness blocker for the swap — the risk is performance, not correctness, and migrating JSEP users -(always-truncating) to native-int64-off is a correctness *upgrade*. - -Follow-up actions (not blockers): - -1. Optionally expose a typed `enableInt64?: boolean` on `WebGpuExecutionProviderOption` for discoverability, plus - document the `extra` opt-in. -2. Benchmark an int64-heavy graph (tokenizer / LLM decode) JSEP vs. native-int64-off to quantify CPU-fallback - cost and confirm the perf trade-off. - -### 6.5 Graph-capture interaction (resolved) - -Graph capture and int64 are **coupled inside the EP**, so there is no conflict to design around. In -`webgpu_execution_provider.cc`: - -```cpp -// enable_int64_ is always true when enable_graph_capture_ is true -enable_int64_{config.enable_graph_capture || config.enable_int64}, -``` - -and the graph-capture kernel registry is built with `RegisterKernels(true, true)` (both flags on). Consequences: - -- **Graph capture OFF (default):** int64 stays off → int64 ops on CPU → full 64-bit correctness. -- **Graph capture ON:** the EP **auto-promotes int64 to ON**, keeping int64 ops on-GPU so the captured region is - all-GPU/capturable. - -So leaving `enableInt64` at its default never breaks graph capture — enabling capture flips int64 on -automatically. Graph-capture users implicitly accept the i32-truncation emulation, but that is exactly the -LLM-decode token-ID regime (`≪ 2³¹`) and is already the status quo today. +**Mechanics.** The native EP reads int64 support from the session-config entry +`ep.webgpuexecutionprovider.enableInt64` (`webgpu_provider_factory.cc`); absent, it defaults to `false`. The web +layer has no typed field for it, but it is reachable today on `/webgpu` via the untyped `extra` map +(`iterateExtraOptions` flattens `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }` into the config key). + +**Emulation (verified).** When int64 is **enabled** on GPU, JSEP and the native EP use identical **i32-truncated** +emulation (storage `vec2`, compute `i32`, high word discarded, 32-bit arithmetic). When int64 is **disabled** +(the native default), ops with int64 *inputs* partition to the CPU/WASM fallback using real `int64_t` — full +64-bit correctness. **Exception:** casting *to* int64 stays on GPU even when int64 is off (the WebGPU `Cast` +kernel only guards int64 *inputs*), so "int64 off" narrows to int64 *inputs*, not all int64 work. + +**Why it's not a blocker.** int64 in real models carries indices/shapes/axes/token IDs, all `≪ 2³¹` (a tensor +with `> 2³¹` elements can't fit in a `< 2GB` buffer), so JSEP (GPU-truncated) and native-int64-off (CPU-full) +produce identical numeric results for realistic models — the difference is **performance only**. `transformers.js` +already runs the native EP with int64 **off** at scale on int64-heavy token-ID models, confirming the +CPU-fallback cost is acceptable. + +**Decision.** Keep int64 **off by default** everywhere, with a uniform native-EP config across `.` and `/webgpu`. +Migrating JSEP users (always-truncating) to native-int64-off is a correctness *upgrade*. Follow-ups (not +blockers): (1) an **optional** typed `enableInt64?: boolean` on `WebGpuExecutionProviderOption` as sugar over the +`extra` key that already works today; (2) benchmark an int64-heavy graph (tokenizer / LLM decode) to quantify the +CPU-fallback cost. + +**Graph-capture interaction (resolved).** Graph capture and int64 are coupled inside the EP +(`enable_int64_{config.enable_graph_capture || config.enable_int64}` in `webgpu_execution_provider.cc`): capture +OFF (default) leaves int64 off (int64 ops on CPU, full correctness); capture ON auto-promotes int64 to keep the +captured region all-GPU/capturable. So the `enableInt64` default never breaks graph capture, and capture users +implicitly accept the i32-truncation regime — exactly the LLM-decode token-ID case (`≪ 2³¹`), the status quo +today. --- ## 7. Open investigation items (to resolve during implementation) -These are potential parity concerns not yet fully verified in source. The first two could be functional/ -correctness blockers and should be checked before the Phase 2 default flip. - -1. **Proxy-worker parity (potential blocker).** `wasm.proxy = true` spins a dedicated worker - (`js/web/lib/wasm/proxy-wrapper.ts`). JSEP runs WebGPU compute over Asyncify; the native EP has its own - threading assumptions. `transformers.js` forces `proxy = false`, so it is **not** prior-art for the proxied - path. Confirm the native EP behaves identically under `wasm.proxy = true`. -2. **IO-binding parity (potential blocker).** Both `'gpu-buffer'` and `'ml-tensor'` output locations are wired in - `js/web/lib/wasm/wasm-core-impl.ts`. Verify the native EP provides the same zero-copy GPU-buffer / ML-tensor - handoff semantics. This will not be caught by op-parity tests. -3. **WebNN native-vs-JSEP parity.** In the default `.` bundle, WebNN is JSEP-hosted; the flip moves those users - to native WebNN as well — a second silent backend swap under the same default bundle. Validate separately from - WebGPU. -4. **Typed-option parity.** Audit whether any typed `WebGpuExecutionProviderOption` field is honored by only one - backend (cache modes, `preferredLayout`, buffer-pool knobs, `forceCpuNodeNames`, etc.). A JSEP-only field - silently becomes a no-op under the native EP. +Parity concerns to close before the Phase 1 flip. Each is an end-to-end validation: the TypeScript wiring is +confirmable by source audit (noted below), but the runtime behavior needs a real browser + GPU/WebNN check. + +1. **Proxy-worker parity (potential blocker).** `wasm.proxy = true` runs compute in a dedicated worker + (`proxy-wrapper.ts`). *Source:* the wrapper is EP-agnostic and already forbids all GPU I/O over the proxy + (CPU-I/O-only for both backends). *Runtime:* verify the native EP's device init + threading work inside a + Worker under Asyncify. (`transformers.js` forces `proxy = false`, so it is not prior art here.) +2. **IO-binding parity (potential blocker).** *Source:* the `'gpu-buffer'` / `'ml-tensor'` paths in + `wasm-core-impl.ts` have symmetric native/JSEP hooks selected by `BUILD_DEFS.DISABLE_WEBGPU` + (`webgpuRegisterBuffer`/`jsepRegisterBuffer`, etc.). *Runtime:* confirm the native WASM exports deliver true + zero-copy handoff and identical dispose/lifetime semantics. Not covered by op-parity tests. +3. **WebNN native-vs-JSEP parity.** The flip also moves default-bundle WebNN from JSEP-hosted to native — a + second silent swap. *Source:* both init paths are visible in `initEp` (`initJsep('webnn', …)` vs + `webnnInit(...)`). *Runtime:* requires a `navigator.ml`-capable environment; E2E-only. + +**Closed by source audit — typed-option parity.** The native EP branch in `session-options.ts` honors a +**superset** of JSEP's per-session options, so no JSEP-only typed field silently becomes a no-op under native +(JSEP's other configuration comes from the global `env.webgpu.*` object, not per-session options). The only +residual is a source check that those global knobs (profiling in particular) reach the native `webgpuInit` path. --- -## 8. Phase 1 — Deprecation (this release) - -No default behavior change. Deliverables: - -1. **JSEP warn-once.** In `wasm-core-impl.ts` `initEp` inside the `if (!BUILD_DEFS.DISABLE_JSEP)` block, for - `epName` `'webgpu'` (and `'webnn'`). Only JSEP builds warn; native-EP builds never warn. -2. **Shared deprecation-warning utility.** Once-only, respects `env.logLevel`, with an opt-out consideration. - Shared with the WebGL-removal effort. -3. **int64 plumbing (prereq).** Plumb `enableInt64` into `js/web/lib/wasm/session-options.ts` (mirroring the - `enableGraphCapture` handling) so migrating users can opt in if they need JSEP-identical GPU int64. -4. **Escape hatch.** Add the temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`), marked - deprecated with a scheduled removal release. -5. **Docs.** Deprecation banners + migration guidance in `js/web/README` and `js/web/docs/webgpu-operators.md`; - release notes / CHANGELOG entries. +## 8. Phase 1 — Flip + deprecate (this release) + +Flip the default to the native EP **and** ship the escape hatch, so the swap is protected by a one-release +fallback. Deliverables: + +1. **Flip the default.** Build the default `.` and `./all` bundles with the native EP (`USE_WEBGPU_EP=true`). The + `webgpu` key and public API are unchanged, so the swap is transparent (§5.1). Gated on the release gate below. +2. **Escape hatch.** Add the temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`), deprecated + with a scheduled removal, shipping in this same release. +3. **JSEP warn-once.** In `wasm-core-impl.ts` `initEp` under `if (!BUILD_DEFS.DISABLE_JSEP)`, for `epName` + `'webgpu'` (and `'webnn'`). Only the `/jsep` build warns; the native default emits nothing. +4. **Shared deprecation-warning utility.** Warn once and respect `env.logLevel` (error/fatal silences it); no + separate opt-out flag. Shared with the WebGL-removal effort. +5. **Publish the tracking issue ahead of release.** The pinned migration issue (§11) ships **before** this release + as the early-warning channel, substituting for a separate warning-only release. Consumers can stay on the + current pre-flip version until ready, then adopt the flip release and pin the `/jsep` hatch it introduces if + needed. +6. **int64 `extra` opt-in (documented).** Document `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }` for + users needing JSEP-identical GPU int64; works today with no new plumbing, typed field optional (§6). +7. **Docs.** Deprecation banners + migration guidance in `js/web/README` and `js/web/docs/webgpu-operators.md`; a + prominent release-notes/CHANGELOG entry (WASM filename `.jsep.wasm` → `.asyncify.wasm`, `/jsep` pin + instructions). Call out that **WebNN** in `.` / `/all` also moves from JSEP-hosted to native WebNN (the same + path `/webgpu` already ships), so WebNN users know to re-test and use `/jsep` on regression — the + WebGPU-centric framing otherwise hides this second swap (§7.3). ### Warning placement rationale -- **Default `.` bundle (still JSEP in Phase 1):** the JSEP warn-once (#1) applies. After the Phase 2 flip it - becomes native EP and emits nothing — the swap is covered in release notes, not a runtime warning, because it - is not actionable for the consumer. -- **`/jsep` escape-hatch bundle:** warns once ("deprecated JSEP build, removed in vX; file an issue if the native - WebGPU EP does not work for you"). Doubles as a parity bug funnel. +The default `.` bundle (native EP) emits nothing — the implementation swap is not actionable for the consumer and +is communicated via the tracking issue and release notes. The `/jsep` escape-hatch bundle warns once ("deprecated +JSEP build, removed in vX; file an issue if the native WebGPU EP does not work for you"), doubling as a parity bug +funnel. ---- +### Release gate -## 9. Phase 2 — Removal (next release) +The default flip (#1) is gated on **both**: -1. **Flip default:** native WebGPU EP becomes the `webgpu` backend in `ort` / `ort.all`; remove the temporary - `USE_WEBGPU_EP` flag. -2. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports. Repoint - `/all` to the webgpu/default artifact (§5.3). -3. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. -4. **Relocate WebNN** out of the `jsep/` directory (`backend-webnn.ts`, `webnn/`) to a neutral path. -5. **Remove `pre-jsep.js` glue;** confirm `post-webgpu.js` / `post-webnn.js` cover all initialization. -6. **Remove the temporary `onnxruntime-web/jsep` export.** +1. **Default-bundle suite runs against the native EP.** CI must exercise the default `.` bundle against the native + EP — a green `/webgpu` run is **not** sufficient, since the bundles differ in wiring (proxy, IO-binding, WebNN + host) even when op kernels match. +2. **§7 blockers resolved with targeted coverage** (op-parity tests don't exercise these paths): the native EP + under `wasm.proxy = true` (§7.1); both `'gpu-buffer'` and `'ml-tensor'` IO-binding zero-copy handoff (§7.2); + native WebNN validated separately (§7.3). Typed-option parity is already closed by source audit; the only + residual is a source check that the global `env.webgpu.*` knobs reach the native `webgpuInit` path. -### Removal ergonomics (no tombstones) +Both gates are required — passing #1 while leaving #2 open would ship acknowledged blocker-class parity gaps +unverified. -The `/jsep` export is removed **cleanly** — no throwing-stub "tombstone" module and no runtime shim are left -behind. A consumer still importing `onnxruntime-web/jsep` after removal gets the native bundler error ("subpath -./jsep is not defined by exports"), which is an acceptable, immediate build-time failure on this temporary, -one-release, opt-in surface. The default `.` import is unaffected (it silently swaps to the native EP), so no -consumer following the documented path hits an error. The removal is communicated via release notes and the -pinned tracking issue rather than a runtime shim. +--- -### Release gate +## 9. Phase 2 — Removal (next release) -Phase 2 (the default flip) is gated on **both** of the following: +The default already runs the native EP (flipped in Phase 1); this release deletes JSEP and the temporary surfaces. -1. **Default-bundle suite runs against the native EP.** The web CI test matrix must exercise the default `.` - bundle against the native WebGPU EP. Today that suite runs JSEP for the `.` bundle; a green `/webgpu` run is - **not** sufficient to prove the default flip, because the two bundles differ in wiring (proxy, IO-binding, - WebNN host) even when op kernels match. -2. **Section 7 blockers resolved with targeted coverage.** The potential blockers in §7 must be closed out before - the flip — not merely tracked. Op-parity tests do **not** exercise these paths, so each needs dedicated - coverage: - - **Proxy-worker path (§7.1):** run the native EP under `wasm.proxy = true` in CI (not just the default - `proxy = false`). - - **IO-binding (§7.2):** add tests for both `'gpu-buffer'` and `'ml-tensor'` output locations that assert the - native EP's zero-copy handoff semantics. - - **Native WebNN (§7.3):** validate the native WebNN path separately from WebGPU, since the default-bundle - WebNN users are silently moved from JSEP-hosted to native WebNN by the same flip. +1. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports, and remove + the temporary `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact (§5.3). +2. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. +3. **Relocate WebNN** out of the `jsep/` directory (`backend-webnn.ts`, `webnn/`) to a neutral path. +4. **Remove `pre-jsep.js` glue;** confirm `post-webgpu.js` / `post-webnn.js` cover all initialization. +5. **Remove the temporary `onnxruntime-web/jsep` export.** - §7.4 (typed-option parity) should be audited in the same pass; a JSEP-only option silently becoming a no-op is - a quieter regression than the first three but is closed the same way (test or documented removal). +### Removal ergonomics (no tombstones) -Passing gate #1 while leaving gate #2 open would let the design meet its stated bar with acknowledged -blocker-class parity gaps still unverified; both are required. +The `/jsep` export is removed cleanly, relying on the native bundler error: a lingering +`import 'onnxruntime-web/jsep'` fails with "subpath ./jsep is not defined by exports" — an acceptable build-time +failure on this temporary, opt-in surface. The default `.` import is unaffected (it already swaps to the native +EP), so no consumer on the documented path hits an error. Communicated via release notes and the tracking issue. --- @@ -326,7 +274,7 @@ blocker-class parity gaps still unverified; both are required. ## 11. Migration guide (for consumers) - **Default import (`onnxruntime-web`):** no code change needed. The `webgpu` backend continues to work; the - implementation swaps to the native EP in the removal release. + implementation swaps to the native EP in Phase 1 (this release). - **`onnxruntime-web/all`:** continues to work and converges to the native EP. - **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). Please file an issue if the native WebGPU EP does not work for your model so the gap can be fixed before JSEP is deleted. @@ -336,17 +284,16 @@ blocker-class parity gaps still unverified; both are required. - **`wasmPaths`:** the backing WASM artifact changes (`.jsep.wasm` → `.asyncify.wasm`); update any hardcoded paths. -> **Canonical source:** the authoritative, continuously-updated migration guidance for both the JSEP and WebGL -> removals lives in a single pinned tracking issue (link TBD). Console warnings, README/docs, and CHANGELOG -> entries **link** to that issue rather than duplicating this guidance, so there is one place to keep current. +> **Canonical source:** this effort has its **own** pinned tracking issue (link TBD) as the authoritative, +> continuously-updated migration guidance, independent of the +> [WebGL-removal](onnxruntime_web_remove_webgl_backend.md) issue. Console warnings, README/docs, and CHANGELOG +> entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) links to +> whichever issue is relevant. --- -## 12. Open questions +## 12. Resolved decisions -1. **Default-flip timing.** Phase 1 vs. Phase 2. Recommendation: Phase 2 (conservative). -2. **Warning suppressibility.** Always-once vs. env opt-out. Recommendation: once + respect `logLevel` so - error/fatal silences it. -3. **WebNN-on-JSEP messaging in Phase 1.** Recommendation: yes — note that `ort` / `ort.all` WebNN moves to the - native path. -4. **int64 typed option in Phase 1.** Recommendation: yes (discoverability), default off. +- **Warning suppressibility.** Warn once and respect `env.logLevel` — setting the log level to error/fatal + silences the deprecation warning; no separate opt-out flag is added. (The WebGL-removal effort adopts the same + policy.) diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index a769bede7b409..f79f95368fd86 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -1,13 +1,12 @@ # Design: Remove the WebGL (onnxjs) backend from onnxruntime-web **Status:** Draft -**Last updated:** 2026-07-10 +**Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGL backend **Related work:** -[Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP](onnxruntime_web_jsep_to_webgpu_ep_migration.md). -The two efforts are independent but share the `onnxruntime-web/all` bundle and the deprecation-warning utility; -see §5 and §6 for the coupling. +[Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP](onnxruntime_web_jsep_to_webgpu_ep_migration.md) +— independent, but shares the `onnxruntime-web/all` bundle and the deprecation-warning utility (see §5–§6). --- @@ -15,15 +14,12 @@ see §5 and §6 for the coupling. `onnxruntime-web` ships a legacy **WebGL** backend — the `onnxjs` implementation under `js/web/lib/onnxjs`, registered under the `'webgl'` backend key. It predates the current WASM architecture, supports a narrower -operator set, and has known fp32/behavioral drift relative to the WebGPU path. +operator set, and has fp32/behavioral drift relative to the WebGPU path. -This document proposes **deprecating and then removing** the WebGL backend. Unlike the JSEP → native WebGPU EP -migration, this is a straightforward **deprecate-and-delete** with an explicit migration message — there is no -transparent redirect (see §5.1). +This document proposes **deprecating and then removing** it as a straightforward **deprecate-and-delete** with an +explicit migration message — no transparent redirect (see §5.1), unlike the JSEP → native WebGPU EP migration: -The work is split into two phases: - -- **Phase 1 (this release):** ship a deprecation warning + docs. No behavior change. +- **Phase 1 (this release):** deprecation warning + docs. No behavior change. - **Phase 2 (next release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` build variant. @@ -74,26 +70,22 @@ nothing. ### 4.1 WebGPU browser coverage (expanding, but with platform caveats) -A common historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that -is **less true than it was**, though MDN's `api.GPU` browser-compat data still reports meaningful caveats — the -Chrome and Firefox entries are **partial**, not blanket support: +The historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that is +**less true**, though MDN's `api.GPU` browser-compat data still shows Chrome and Firefox as **partial**: | Browser | WebGPU (per MDN BCD) | |---|---| -| Chrome / Edge (desktop) | Partial from 113 (ChromeOS/macOS/Windows); broader/full only in a later version, with Linux limited to newer Intel-gen GPUs | +| Chrome / Edge (desktop) | Partial from 113 (ChromeOS/macOS/Windows); Linux limited to newer Intel-gen GPUs | | Chrome Android | 121+ | -| Safari (macOS) | Full, 26 | -| Safari (iOS) | Full, 26 | +| Safari (macOS / iOS) | Full, 26 | | Chrome / Edge (iOS, via WebKit) | Full, 26 | | Firefox (desktop) | Partial from 141; **excludes Linux and Intel-based macOS** (Apple-Silicon-first) | | Firefox for Android | ❌ | -So Safari/iOS coverage is now solid, and Chrome/Firefox are moving in the right direction — but there remain real -gaps: older browser versions, Firefox for Android, Linux (both Chrome's GPU-generation limits and Firefox's -exclusion), and Intel-based Macs on Firefox. WebGPU coverage is **broadening but not yet universal**, so the set -of users for whom WebGL is the *only* GPU path is shrinking rather than gone. This supports removing WebGL over a -deprecation window, but the design does **not** rely on WebGPU being universally available — the actionable -fallback for uncovered users is the WASM/CPU backend, not WebGPU. +Safari/iOS is now solid and Chrome/Firefox are improving, but real gaps remain (older versions, Firefox for +Android, Linux, Intel Macs on Firefox). Coverage is **broadening but not universal**, so the pool of users for +whom WebGL is the *only* GPU path is shrinking, not gone. The design does **not** assume universal WebGPU — the +actionable fallback for uncovered users is the WASM/CPU backend. --- @@ -101,45 +93,40 @@ fallback for uncovered users is the WASM/CPU backend, not WebGPU. ### 5.1 Explicit removal, not a transparent redirect -Unlike the WebGPU JSEP → native swap, WebGL cannot be transparently redirected to another backend: +WebGL cannot be transparently redirected (unlike the WebGPU JSEP → native swap): -- The `./webgl` bundle sets `DISABLE_WASM: true`, so there is **no WASM/CPU fallback** to silently fall back to. -- WebGPU requires `navigator.gpu`; a silent redirect would fail on devices without WebGPU support. -- WebGL exhibits fp32/op drift, so silently swapping results would be a hidden behavioral change. +- The `./webgl` bundle sets `DISABLE_WASM: true`, so there is **no WASM/CPU fallback** to redirect to. +- WebGPU requires `navigator.gpu`; a silent redirect would fail where WebGPU is unavailable. +- WebGL's fp32/op drift means silently swapping results would be a hidden behavioral change. -Therefore WebGL gets an **explicit deprecation warning + removal** with an actionable migration message directing -users to the WebGPU EP (`webgpu`) or the WASM/CPU backend (`wasm`). +So WebGL gets an **explicit deprecation warning + removal** directing users to the WebGPU EP (`webgpu`) or the +WASM/CPU backend (`wasm`). ### 5.2 Warning placement The warning fires in `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), in `init()` / -`createInferenceSessionHandler`. Because the backend registers with negative priority, this only fires when WebGL -is **explicitly requested**, so it does not spam consumers who never opt into WebGL. It fires for both `ort.all` -and `ort.webgl`. +`createInferenceSessionHandler`. Because the backend has negative priority, it only fires when WebGL is +**explicitly requested** — no spam for consumers who never opt in. Fires for both `ort.all` and `ort.webgl`. ### 5.3 Removal ergonomics (no tombstones) -When the backend is removed in Phase 2, we do **not** ship throwing-stub "tombstone" modules for the `./webgl` -export or a runtime shim for the `'webgl'` backend key. The rationale: +Phase 2 relies on the native failure modes rather than adding tombstone stubs or a `'webgl'`-key shim: + +- A lingering `import 'onnxruntime-web/webgl'` fails with the native bundler error ("subpath ./webgl is not + defined by exports") — a clear build-time failure. +- `executionProviders: ['webgl']` throws "no available backend found" from `resolveBackendAndExecutionProviders`. -- Removing the `./webgl` export means a lingering `import 'onnxruntime-web/webgl'` fails with the native bundler - error ("subpath ./webgl is not defined by exports"). That is a clear, immediate, build-time failure — no shim - needed. -- Requesting `executionProviders: ['webgl']` alone after removal already throws the generic "no available backend - found" error from `resolveBackendAndExecutionProviders`. -- WebGL was never a production-grade, first-class backend (narrow op set, fp32 drift, negative priority), so an - elaborate soft-landing is not warranted. +WebGL was never a first-class backend (narrow op set, fp32 drift, negative priority), so these built-in errors are +a sufficient soft-landing. --- ## 6. `/all` bundle coupling -WebGL is one of the two backends in `./all` (the other being JSEP WebGPU/WebNN). Removing WebGL drops it from -`/all`, leaving JSEP (which the -[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) subsequently converges to the native -EP). This effort owns **removing WebGL from `/all`**; the JSEP effort owns the **final convergence** of `/all` -onto the native artifact. `/all` itself is **kept** as an export to avoid breaking imports (see the JSEP doc §5.3 -for the alias decision). +`./all` holds two backends: WebGL and JSEP WebGPU/WebNN. This effort owns **removing WebGL from `/all`**; the +[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) owns the **final convergence** of the +remaining JSEP backend onto the native EP. `/all` is **kept** as an export to avoid breaking imports (see JSEP doc +§5.3 for the alias decision). --- @@ -184,15 +171,19 @@ No behavior change. Deliverables: - **`onnxruntime-web/all`:** continues to work, but no longer includes WebGL. If you relied on WebGL as a fallback via `/all`, add `wasm` to your `executionProviders` list. -> **Canonical source:** the authoritative, continuously-updated migration guidance for both the WebGL and JSEP -> removals lives in a single pinned tracking issue (link TBD). Console warnings, README/docs, and CHANGELOG -> entries **link** to that issue rather than duplicating this guidance, so there is one place to keep current. +> **Canonical source:** this effort has its **own** pinned tracking issue (link TBD) as the authoritative, +> continuously-updated migration guidance, independent of the +> [JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) issue. Console warnings, README/docs, +> and CHANGELOG entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) +> links to whichever issue is relevant. --- -## 11. Open questions +## 11. Resolved decisions -1. **Warning suppressibility.** Always-once vs. env opt-out. Recommendation: once + respect `logLevel` so - error/fatal silences it. (Consistent with the JSEP effort.) -2. **Deprecation window length.** One release (aligned with the JSEP removal) vs. longer. Recommendation: align - with the JSEP removal for a single coordinated backend-consolidation message. +- **Deprecation window length.** Default to **one release**, kept **independent** of the JSEP removal's schedule + rather than forcing the two to align. Extend only if issues surface during deprecation (e.g. WebGL-reliant + consumers report friction migrating to `wasm`/WebGPU) — real-world feedback, not the JSEP timeline, drives any + extension. +- **Warning suppressibility.** Warn once and respect `env.logLevel` — setting the log level to error/fatal + silences the deprecation warning; no separate opt-out flag is added. (Same policy as the JSEP effort.) From 3788336138e3f6228dff7faa72f51f1d3c711db1 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Tue, 14 Jul 2026 12:39:25 -0700 Subject: [PATCH 04/10] Reconcile /all sequencing and refine onnxruntime-web backend deprecation docs Make the /all bundle convergence order-independent across both design docs: the JSEP->native flip and the WebGL removal are independent efforts that may land in either order or different releases, and /all collapses into a single physical alias only once both have landed (whichever lands second performs the repoint). Also: elevate the global env.webgpu.* settings parity gap to a prerequisite (JSEP open item, release gate, and risk row); correct the WebGPU browser-coverage table to MDN's current data (Chrome/Edge full from 144); fold the WebGL deprecation-window decision into the Phase 2 section; drop the Status: Draft lines and the redundant Resolved-decisions sections; and align Phase 2 wording (subsequent release). Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 80 +++++++++++-------- .../onnxruntime_web_remove_webgl_backend.md | 61 +++++++------- 2 files changed, 74 insertions(+), 67 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index ccbf071c3600a..0a4cd52bb2dfc 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -1,6 +1,5 @@ # Design: Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP -**Status:** Draft **Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends @@ -27,7 +26,7 @@ undiscovered parity gaps. `onnxruntime-web/jsep` escape-hatch export, and ship deprecation warnings + docs. The flip and the hatch ship together, so the native default and its JSEP fallback coexist for exactly one release. The pinned tracking issue is published **ahead of** this release as the early-warning channel — no separate warning-only release (§8). -- **Phase 2 (next release):** delete JSEP and remove the temporary `/jsep` export and build flags. +- **Phase 2 (subsequent release):** delete JSEP and remove the temporary `/jsep` export and build flags. --- @@ -115,18 +114,22 @@ deleted. ### 5.3 `/all` bundle: keep as a converging alias -`./all` today differs from the default only by including WebGL. After the WebGL removal (tracked -[separately](onnxruntime_web_remove_webgl_backend.md)) and this JSEP → native swap, it converges to **native -WebGPU EP + native WebNN**, identical to `./webgpu` and the default `.`. +`./all` today bundles two **independent** things: the WebGPU/WebNN backend (JSEP) **and** WebGL. Two independent +efforts change it — this migration flips its WebGPU/WebNN half from JSEP to the native EP, and the +[WebGL removal](onnxruntime_web_remove_webgl_backend.md) drops WebGL. Once **both** have landed, `/all` converges +to **native WebGPU EP + native WebNN**, identical to `./webgpu` and the default `.`. -**Decision:** keep `/all` to avoid breaking imports, implemented as a **real alias to the same physical artifact** -as the webgpu/default bundle rather than a separately-built `ort.all.*` — avoiding bundle drift, dropping a build -target, and not doubling the CDN/WASM payload. The backing WASM shifts `.jsep.wasm` → `.asyncify.wasm` with the -general filename migration. "all" becomes a mild misnomer (no longer implies WebGL) — documented, not worth an -export break. +**Decision:** keep `/all` to avoid breaking imports. Its **end state** is a **real alias to the same physical +artifact** as the webgpu/default bundle (rather than a separately-built `ort.all.*`) — avoiding bundle drift, +dropping a build target, and not doubling the CDN/WASM payload. The backing WASM shifts `.jsep.wasm` → +`.asyncify.wasm` with the general filename migration. "all" becomes a mild misnomer (no longer implies WebGL) — +documented, not worth an export break. -**Sequencing:** the WebGL effort drops WebGL from `/all` first (still JSEP); this swap then converges `/all` onto -the native artifact. Each effort owns its half. +**Sequencing (order-independent).** The native flip (this effort) and the WebGL drop (the WebGL effort) are +independent and may land in either order, or in different releases. Until both land, `/all` stays a **distinct +bundle**: after this effort's Phase 1 flip but before WebGL is removed, `/all` is **native WebGPU/WebNN + WebGL** +(still not identical to default/`./webgpu`). Whichever effort lands second collapses `/all` into the physical +alias — see §9 (this effort's Phase 2) and the WebGL doc §8. --- @@ -166,8 +169,9 @@ today. ## 7. Open investigation items (to resolve during implementation) -Parity concerns to close before the Phase 1 flip. Each is an end-to-end validation: the TypeScript wiring is -confirmable by source audit (noted below), but the runtime behavior needs a real browser + GPU/WebNN check. +Parity concerns to close before the Phase 1 flip. Most are end-to-end validations — the TypeScript wiring is +confirmable by source audit (noted below), but the runtime behavior needs a real browser + GPU/WebNN check. Item +4 is not a runtime unknown but a known, addressable wiring gap — a prerequisite, not a blocker. 1. **Proxy-worker parity (potential blocker).** `wasm.proxy = true` runs compute in a dedicated worker (`proxy-wrapper.ts`). *Source:* the wrapper is EP-agnostic and already forbids all GPU I/O over the proxy @@ -180,11 +184,21 @@ confirmable by source audit (noted below), but the runtime behavior needs a real 3. **WebNN native-vs-JSEP parity.** The flip also moves default-bundle WebNN from JSEP-hosted to native — a second silent swap. *Source:* both init paths are visible in `initEp` (`initJsep('webnn', …)` vs `webnnInit(...)`). *Runtime:* requires a `navigator.ml`-capable environment; E2E-only. - -**Closed by source audit — typed-option parity.** The native EP branch in `session-options.ts` honors a -**superset** of JSEP's per-session options, so no JSEP-only typed field silently becomes a no-op under native -(JSEP's other configuration comes from the global `env.webgpu.*` object, not per-session options). The only -residual is a source check that those global knobs (profiling in particular) reach the native `webgpuInit` path. +4. **Global `env.webgpu.*` settings parity (prerequisite).** A known, addressable gap: the default bundle's + global WebGPU knobs are **not** honored by the native EP today. *Source:* `wasm-core-impl.ts` reads + `env.webgpu.adapter` / `powerPreference` / `forceFallbackAdapter` only to call `navigator.gpu.requestAdapter()` + on the JS side, then calls `webgpuInit()` **without passing that adapter** to the native EP — the JSEP branch + does forward it to `initJsep`, so the native path drops it. The C++ factory has a `powerPreference` config key + (`webgpu_provider_factory.cc`) that the TS layer never populates, and `startProfiling()` in + `session-handler-inference.ts` is still a `// TODO` on the native path while JSEP consumes + `env.webgpu.profiling.ondata`. So after the flip, consumers relying on these globals silently lose them. + *Action (required before the flip):* either wire the relevant settings into the native path (or typed + per-session replacements), or explicitly document the dropped/changed settings with migration guidance — a + release-gate item (§8). + +**Closed by source audit — per-session typed-option parity.** The native EP branch in `session-options.ts` honors +a **superset** of JSEP's per-session typed options, so no JSEP-only typed field silently becomes a no-op under +native. JSEP's *global* `env.webgpu.*` configuration is a separate, still-open gap — see item 4. --- @@ -194,7 +208,8 @@ Flip the default to the native EP **and** ship the escape hatch, so the swap is fallback. Deliverables: 1. **Flip the default.** Build the default `.` and `./all` bundles with the native EP (`USE_WEBGPU_EP=true`). The - `webgpu` key and public API are unchanged, so the swap is transparent (§5.1). Gated on the release gate below. + `webgpu` key and public API are unchanged, so the swap is transparent (§5.1). `/all` still bundles WebGL until + the WebGL effort removes it, so it stays a distinct bundle for now (§5.3). Gated on the release gate below. 2. **Escape hatch.** Add the temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`), deprecated with a scheduled removal, shipping in this same release. 3. **JSEP warn-once.** In `wasm-core-impl.ts` `initEp` under `if (!BUILD_DEFS.DISABLE_JSEP)`, for `epName` @@ -229,26 +244,30 @@ The default flip (#1) is gated on **both**: host) even when op kernels match. 2. **§7 blockers resolved with targeted coverage** (op-parity tests don't exercise these paths): the native EP under `wasm.proxy = true` (§7.1); both `'gpu-buffer'` and `'ml-tensor'` IO-binding zero-copy handoff (§7.2); - native WebNN validated separately (§7.3). Typed-option parity is already closed by source audit; the only - residual is a source check that the global `env.webgpu.*` knobs reach the native `webgpuInit` path. + native WebNN validated separately (§7.3); and the global `env.webgpu.*` settings gap (§7.4) closed by wiring + `adapter` / `powerPreference` / `forceFallbackAdapter` / `profiling` into the native path (or typed per-session + replacements), or by documenting the dropped/changed settings with migration guidance. Per-session + typed-option parity is already closed by source audit. Both gates are required — passing #1 while leaving #2 open would ship acknowledged blocker-class parity gaps unverified. --- -## 9. Phase 2 — Removal (next release) +## 9. Phase 2 — Removal (subsequent release) The default already runs the native EP (flipped in Phase 1); this release deletes JSEP and the temporary surfaces. 1. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports, and remove - the temporary `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact (§5.3). + the temporary `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact (§5.3) — valid only once + WebGL has also been dropped from `/all` (WebGL doc §8); if WebGL is still present, `/all` stays a distinct + bundle and the repoint waits on that removal. 2. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. 3. **Relocate WebNN** out of the `jsep/` directory (`backend-webnn.ts`, `webnn/`) to a neutral path. 4. **Remove `pre-jsep.js` glue;** confirm `post-webgpu.js` / `post-webnn.js` cover all initialization. 5. **Remove the temporary `onnxruntime-web/jsep` export.** -### Removal ergonomics (no tombstones) +### Removal ergonomics The `/jsep` export is removed cleanly, relying on the native bundler error: a lingering `import 'onnxruntime-web/jsep'` fails with "subpath ./jsep is not defined by exports" — an acceptable build-time @@ -266,12 +285,13 @@ EP), so no consumer on the documented path hits an error. Communicated via relea | Proxy-worker path behaves differently | Unknown | **Investigate before flip** (§7.1) | | IO-binding (gpu-buffer/ml-tensor) semantics differ | Unknown | **Investigate before flip** (§7.2) | | WebNN behavior changes under native path | Unknown | Validate native WebNN separately (§7.3) | +| Global `env.webgpu.*` settings (adapter/powerPreference/forceFallbackAdapter/profiling) silently dropped on native | Medium | Wire into the native path or document dropped/changed settings with migration guidance — release gate (§7.4, §8) | | WASM artifact filename change breaks `wasmPaths` | Medium | Document migration; call out `.jsep.wasm` → `.asyncify.wasm` | | No telemetry on JSEP adoption | Medium | Warn-once funnel + one-release escape hatch as the feedback mechanism | --- -## 11. Migration guide (for consumers) +## 11. Migration guide - **Default import (`onnxruntime-web`):** no code change needed. The `webgpu` backend continues to work; the implementation swaps to the native EP in Phase 1 (this release). @@ -289,11 +309,3 @@ EP), so no consumer on the documented path hits an error. Communicated via relea > [WebGL-removal](onnxruntime_web_remove_webgl_backend.md) issue. Console warnings, README/docs, and CHANGELOG > entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) links to > whichever issue is relevant. - ---- - -## 12. Resolved decisions - -- **Warning suppressibility.** Warn once and respect `env.logLevel` — setting the log level to error/fatal - silences the deprecation warning; no separate opt-out flag is added. (The WebGL-removal effort adopts the same - policy.) diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index f79f95368fd86..1dabd06148e80 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -1,6 +1,5 @@ # Design: Remove the WebGL (onnxjs) backend from onnxruntime-web -**Status:** Draft **Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGL backend @@ -20,7 +19,7 @@ This document proposes **deprecating and then removing** it as a straightforward explicit migration message — no transparent redirect (see §5.1), unlike the JSEP → native WebGPU EP migration: - **Phase 1 (this release):** deprecation warning + docs. No behavior change. -- **Phase 2 (next release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` build +- **Phase 2 (subsequent release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` build variant. --- @@ -68,30 +67,29 @@ It appears in two bundles: The `./webgl` bundle sets `BUILD_DEFS.DISABLE_WASM: true`, so it has **no WASM/CPU fallback** — it is WebGL or nothing. -### 4.1 WebGPU browser coverage (expanding, but with platform caveats) +### 4.1 WebGPU browser coverage The historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that is -**less true**, though MDN's `api.GPU` browser-compat data still shows Chrome and Firefox as **partial**: +**less true**, though per MDN's `api.GPU` browser-compat data gaps remain: | Browser | WebGPU (per MDN BCD) | |---|---| -| Chrome / Edge (desktop) | Partial from 113 (ChromeOS/macOS/Windows); Linux limited to newer Intel-gen GPUs | -| Chrome Android | 121+ | -| Safari (macOS / iOS) | Full, 26 | -| Chrome / Edge (iOS, via WebKit) | Full, 26 | -| Firefox (desktop) | Partial from 141; **excludes Linux and Intel-based macOS** (Apple-Silicon-first) | +| Chrome / Edge (desktop) | Full from 144 (Linux: Intel Gen12+ only) | +| Chrome Android | Full from 121 | +| Safari (macOS / iOS) | Full from 26 | +| Firefox (desktop) | Partial from 141 | | Firefox for Android | ❌ | -Safari/iOS is now solid and Chrome/Firefox are improving, but real gaps remain (older versions, Firefox for -Android, Linux, Intel Macs on Firefox). Coverage is **broadening but not universal**, so the pool of users for -whom WebGL is the *only* GPU path is shrinking, not gone. The design does **not** assume universal WebGPU — the -actionable fallback for uncovered users is the WASM/CPU backend. +Safari/iOS and Chrome/Edge desktop are solid, but gaps remain (older versions, Firefox for Android, Firefox +desktop still partial). Coverage is **broadening but not universal**, so the pool of users for whom WebGL is the +*only* GPU path is shrinking, not gone. The design does **not** assume universal WebGPU — the actionable fallback +for uncovered users is the WASM/CPU backend. --- ## 5. Proposed approach -### 5.1 Explicit removal, not a transparent redirect +### 5.1 Explicit removal WebGL cannot be transparently redirected (unlike the WebGPU JSEP → native swap): @@ -108,7 +106,7 @@ The warning fires in `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), in `init( `createInferenceSessionHandler`. Because the backend has negative priority, it only fires when WebGL is **explicitly requested** — no spam for consumers who never opt in. Fires for both `ort.all` and `ort.webgl`. -### 5.3 Removal ergonomics (no tombstones) +### 5.3 Removal ergonomics Phase 2 relies on the native failure modes rather than adding tombstone stubs or a `'webgl'`-key shim: @@ -123,10 +121,12 @@ a sufficient soft-landing. ## 6. `/all` bundle coupling -`./all` holds two backends: WebGL and JSEP WebGPU/WebNN. This effort owns **removing WebGL from `/all`**; the -[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) owns the **final convergence** of the -remaining JSEP backend onto the native EP. `/all` is **kept** as an export to avoid breaking imports (see JSEP doc -§5.3 for the alias decision). +`./all` bundles two **independent** things: WebGL **and** the WebGPU/WebNN backend. This effort owns **removing +WebGL from `/all`**; the [JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) owns flipping +the WebGPU/WebNN half from JSEP to the native EP. The two changes are independent and may land in either order, or +in different releases; `/all` collapses into a single physical alias of the webgpu/default bundle only once +**both** have landed (whichever lands second performs the repoint). `/all` is **kept** as an export to avoid +breaking imports (see JSEP doc §5.3 for the alias decision). --- @@ -142,13 +142,19 @@ No behavior change. Deliverables: --- -## 8. Phase 2 — Removal (next release) +## 8. Phase 2 — Removal (subsequent release) + +Timed a **single release** after Phase 1 by default, kept **independent** of the JSEP removal's schedule rather +than forcing the two to align. Extend only if issues surface during deprecation (e.g. WebGL-reliant consumers +report friction migrating to `wasm`/WebGPU) — real-world feedback, not the JSEP timeline, drives any extension. 1. **Delete WebGL:** remove the `onnxjs` backend (`js/web/lib/onnxjs`), the `'webgl'` registration, and `backend-onnxjs.ts`. 2. **Remove the `ort.webgl` build variant;** update `build.ts` and `package.json` exports (remove the `./webgl` export). -3. **Drop WebGL from `ort.all`** (see §6). +3. **Drop WebGL from `ort.all`** (see §6). If the JSEP → native flip has already landed, this is the step that + collapses `/all` into the webgpu/default alias; if not, `/all` remains a distinct JSEP WebGPU/WebNN bundle + until that flip lands. 4. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_WEBGL` and the code it gates; simplify `index.ts`. --- @@ -163,7 +169,7 @@ No behavior change. Deliverables: --- -## 10. Migration guide (for consumers) +## 10. Migration guide - **`onnxruntime-web/webgl`:** removed. Migrate to the WebGPU EP (`executionProviders: ['webgpu']`) or the WASM/CPU backend (`executionProviders: ['wasm']`). There is no automatic redirect because WebGL builds have no @@ -176,14 +182,3 @@ No behavior change. Deliverables: > [JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) issue. Console warnings, README/docs, > and CHANGELOG entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) > links to whichever issue is relevant. - ---- - -## 11. Resolved decisions - -- **Deprecation window length.** Default to **one release**, kept **independent** of the JSEP removal's schedule - rather than forcing the two to align. Extend only if issues surface during deprecation (e.g. WebGL-reliant - consumers report friction migrating to `wasm`/WebGPU) — real-world feedback, not the JSEP timeline, drives any - extension. -- **Warning suppressibility.** Warn once and respect `env.logLevel` — setting the log level to error/fatal - silences the deprecation warning; no separate opt-out flag is added. (Same policy as the JSEP effort.) From e3c889f7ef7f4d4a8540fcd2a8419b42513e8387 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Tue, 14 Jul 2026 12:45:34 -0700 Subject: [PATCH 05/10] Document JSEP deprecation-window extension in Phase 2 Mirror the WebGL doc: note the single-release default for the /jsep escape hatch and that extension is driven by native-EP parity regressions surfacing via /jsep fallback usage, not a fixed calendar. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index 0a4cd52bb2dfc..99a599e96a936 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -258,6 +258,10 @@ unverified. The default already runs the native EP (flipped in Phase 1); this release deletes JSEP and the temporary surfaces. +Timed a **single release** after Phase 1 by default, keeping the `/jsep` escape hatch (and thus the JSEP +implementation) available for exactly one release. Extend only if native-EP parity regressions surface in the wild +— real fallback usage via `/jsep`, not a fixed calendar, drives any extension. + 1. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports, and remove the temporary `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact (§5.3) — valid only once WebGL has also been dropped from `/all` (WebGL doc §8); if WebGL is still present, `/all` stays a distinct From a358c0a1a2c803338cf55b0264e90ae48e7647fe Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Wed, 15 Jul 2026 10:46:50 -0700 Subject: [PATCH 06/10] Address PR 29716 review feedback on onnxruntime-web deprecation docs - WebGPU coverage table (WebGL doc 4.1): switch from pinned MDN 'full support' versions to qualitative status, note WebGPU has shipped by default in Chrome/Edge since 113, keep Firefox as plain 'partial' (no roadmap claim), and link MDN api.GPU as the live source (per Copilot reviewer). - Migration guide (JSEP doc 11): add guidance to prefer onnxruntime-web/jspi on JSPI-capable browsers for a smaller WASM and lower per-call overhead, with Asyncify as the universal fallback (per qjia7). - int64 gap (JSEP doc 6): replace the misleading 'CPU-fallback cost' phrasing with the actual cost (CPU/WASM execution of int64-input ops plus partition-boundary copies; no GPU-to-CPU download of int64 data) (per qjia7). - Remove the Last updated field from both docs. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 14 ++++++---- .../onnxruntime_web_remove_webgl_backend.md | 28 ++++++++++--------- 2 files changed, 24 insertions(+), 18 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index 99a599e96a936..147f99aae7cac 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -1,6 +1,5 @@ # Design: Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP -**Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends **Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md) @@ -149,14 +148,15 @@ kernel only guards int64 *inputs*), so "int64 off" narrows to int64 *inputs*, no **Why it's not a blocker.** int64 in real models carries indices/shapes/axes/token IDs, all `≪ 2³¹` (a tensor with `> 2³¹` elements can't fit in a `< 2GB` buffer), so JSEP (GPU-truncated) and native-int64-off (CPU-full) produce identical numeric results for realistic models — the difference is **performance only**. `transformers.js` -already runs the native EP with int64 **off** at scale on int64-heavy token-ID models, confirming the -CPU-fallback cost is acceptable. +already runs the native EP with int64 **off** at scale on int64-heavy token-ID models, confirming the overhead of +running those int64-input ops on CPU/WASM (plus the partition-boundary copies — int64 inputs typically originate on +CPU, and results are uploaded back to the GPU) is acceptable. **Decision.** Keep int64 **off by default** everywhere, with a uniform native-EP config across `.` and `/webgpu`. Migrating JSEP users (always-truncating) to native-int64-off is a correctness *upgrade*. Follow-ups (not blockers): (1) an **optional** typed `enableInt64?: boolean` on `WebGpuExecutionProviderOption` as sugar over the -`extra` key that already works today; (2) benchmark an int64-heavy graph (tokenizer / LLM decode) to quantify the -CPU-fallback cost. +`extra` key that already works today; (2) benchmark an int64-heavy graph (tokenizer / LLM decode) to quantify that +cost (CPU/WASM execution of the int64-input ops plus the partition-boundary copies). **Graph-capture interaction (resolved).** Graph capture and int64 are coupled inside the EP (`enable_int64_{config.enable_graph_capture || config.enable_int64}` in `webgpu_execution_provider.cc`): capture @@ -300,6 +300,10 @@ EP), so no consumer on the documented path hits an error. Communicated via relea - **Default import (`onnxruntime-web`):** no code change needed. The `webgpu` backend continues to work; the implementation swaps to the native EP in Phase 1 (this release). - **`onnxruntime-web/all`:** continues to work and converges to the native EP. +- **Lower-overhead build (`onnxruntime-web/jspi`):** when you can target JSPI-capable browsers, prefer the `./jspi` + bundle. It runs the **same native WebGPU EP** but bridges async via **JSPI** instead of Asyncify, giving a + smaller WASM binary and lower per-call overhead. The default (Asyncify) build remains the universal fallback for + browsers without JSPI. - **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). Please file an issue if the native WebGPU EP does not work for your model so the gap can be fixed before JSEP is deleted. - **int64-heavy models needing JSEP-identical GPU behavior:** set diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index 1dabd06148e80..cb033c4be5c3f 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -1,6 +1,5 @@ # Design: Remove the WebGL (onnxjs) backend from onnxruntime-web -**Last updated:** 2026-07-14 **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGL backend **Related work:** @@ -70,20 +69,23 @@ nothing. ### 4.1 WebGPU browser coverage The historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that is -**less true**, though per MDN's `api.GPU` browser-compat data gaps remain: +**less true** — WebGPU has shipped enabled-by-default in Chrome/Edge since **113** (2023), in Safari (macOS/iOS), +and in Chrome for Android; Firefox support is partial. Per MDN's +[`api.GPU`](https://developer.mozilla.org/docs/Web/API/GPU#browser_compatibility) browser-compat data, gaps +remain: -| Browser | WebGPU (per MDN BCD) | +| Browser | WebGPU | |---|---| -| Chrome / Edge (desktop) | Full from 144 (Linux: Intel Gen12+ only) | -| Chrome Android | Full from 121 | -| Safari (macOS / iOS) | Full from 26 | -| Firefox (desktop) | Partial from 141 | -| Firefox for Android | ❌ | - -Safari/iOS and Chrome/Edge desktop are solid, but gaps remain (older versions, Firefox for Android, Firefox -desktop still partial). Coverage is **broadening but not universal**, so the pool of users for whom WebGL is the -*only* GPU path is shrinking, not gone. The design does **not** assume universal WebGPU — the actionable fallback -for uncovered users is the WASM/CPU backend. +| Chrome / Edge (desktop) | ✅ (default since 113; Linux support landed later) | +| Chrome (Android) | ✅ | +| Safari (macOS / iOS) | ✅ | +| Firefox (desktop) | ⚠️ partial | +| Firefox (Android) | ❌ | + +Chrome/Edge and Safari/iOS are solid, but gaps remain (older versions, Firefox for Android, Firefox desktop still +partial). Coverage is **broadening but not universal**, so the pool of users for whom WebGL is the *only* GPU path +is shrinking, not gone. The design does **not** assume universal WebGPU — the actionable fallback for uncovered +users is the WASM/CPU backend. --- From fe0516f9055f6a5ceee7f9eda7a5fe33c9b650a9 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Fri, 31 Jul 2026 12:19:26 -0700 Subject: [PATCH 07/10] Simplify onnxruntime-web backend deprecation design docs Rewrite both design docs to read as clean decision docs rather than a record of the review discussion: drop meta-labels (verified/resolved/red herring), collapse repeated justifications, condense the int64 and open-items sections, and trim the risk tables and migration guides. Keep all concrete technical facts (file names, BUILD_DEFS flags, config keys, phased plan) and fix cross-references after collapsing subsections. Also refine the int64 framing: drop the 'never a regression / enhancement' verdict and describe enableInt64=1 as an opt-in speed/precision tradeoff (lossy for genuine > 2^31 values). Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 344 ++++++------------ .../onnxruntime_web_remove_webgl_backend.md | 128 +++---- 2 files changed, 162 insertions(+), 310 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index 147f99aae7cac..0d8a5918bf735 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -3,7 +3,7 @@ **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends **Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md) -— independent, but shares the `onnxruntime-web/all` bundle and the deprecation-warning utility (see §5.3, §8). +— independent, but shares the `onnxruntime-web/all` bundle and the deprecation-warning utility. --- @@ -11,33 +11,27 @@ `onnxruntime-web` implements WebGPU/WebNN compute two ways: -- **JSEP** — WebGPU (and WebNN) compute implemented in TypeScript over an Asyncify-compiled WASM core. Powers the - current default (`.`) and `./all` bundles. -- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM (a port/superset of JSEP). Powers the - current `./webgpu` and `./jspi` bundles. +- **JSEP** — WebGPU/WebNN implemented in TypeScript over an Asyncify-compiled WASM core. Powers the default (`.`) + and `./all` bundles. +- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM. Powers the `./webgpu` and `./jspi` + bundles. -This document proposes **deprecating and removing JSEP**, standardizing on the **native WebGPU EP** as the single -WebGPU/WebNN path. The swap is **transparent** for the common case — the default bundle keeps the `webgpu` backend -key and changes implementation at build time — with a **temporary one-release escape hatch** as insurance against -undiscovered parity gaps. +This document proposes removing JSEP and standardizing on the native WebGPU EP. The default bundle keeps the +`webgpu` backend key and swaps implementation at build time, so the change is transparent for existing code. -- **Phase 1 (this release):** flip the default (`.`) and `./all` bundles to the native EP, add the temporary - `onnxruntime-web/jsep` escape-hatch export, and ship deprecation warnings + docs. The flip and the hatch ship - together, so the native default and its JSEP fallback coexist for exactly one release. The pinned tracking issue - is published **ahead of** this release as the early-warning channel — no separate warning-only release (§8). +- **Phase 1 (this release):** flip the default (`.`) and `./all` bundles to the native EP, add a temporary + `onnxruntime-web/jsep` escape-hatch export for one release, and ship deprecation warnings + docs. - **Phase 2 (subsequent release):** delete JSEP and remove the temporary `/jsep` export and build flags. --- ## 2. Motivation -- **Duplication of maintenance.** JSEP and the native WebGPU EP implement the same operators twice — once in - TypeScript and once in C++. The native EP is the strategic direction and receives the bulk of new op work; - JSEP is effectively a second implementation that must be kept in lockstep. -- **Single, consistent runtime semantics.** Consolidating on the native EP means the browser build shares kernel - behavior with the rest of ONNX Runtime rather than maintaining a parallel TS implementation. -- **Smaller/simpler build matrix.** Removing JSEP lets us delete the temporary `USE_WEBGPU_EP` / `DISABLE_JSEP` - build plumbing. +- **Remove duplicate maintenance.** JSEP and the native EP implement the same operators twice (TypeScript and + C++). The native EP is the strategic direction and receives new op work. +- **Consistent runtime semantics.** The browser build shares kernel behavior with the rest of ONNX Runtime + instead of maintaining a parallel TS implementation. +- **Simpler build matrix.** Removing JSEP deletes the temporary `USE_WEBGPU_EP` / `DISABLE_JSEP` build plumbing. --- @@ -45,35 +39,26 @@ undiscovered parity gaps. ### Goals -- Make the native WebGPU EP the default WebGPU/WebNN backend for `onnxruntime-web`. -- Do so **transparently** for the common consumer (same import, same `webgpu` backend key, same public API). -- Give existing JSEP consumers a clearly-communicated, low-effort migration path plus a one-release safety net. +- Make the native WebGPU EP the default WebGPU/WebNN backend, transparently for existing consumers (same import, + `webgpu` key, and public API). +- Give JSEP consumers a low-effort migration path plus a one-release safety net. - Remove the JSEP TypeScript compute path and its build variants. ### Non-goals -- **WebGL removal** — tracked separately in - [onnxruntime_web_remove_webgl_backend.md](onnxruntime_web_remove_webgl_backend.md). -- **No change to the Node binding** (`onnxruntime-node` / `ort.node.*`). -- **No change to the React-Native package.** -- **No change to ORT training-web.** -- Not introducing new WebGPU features — this is a consolidation, not a feature effort. +- **WebGL removal** — tracked in [onnxruntime_web_remove_webgl_backend.md](onnxruntime_web_remove_webgl_backend.md). +- No change to the Node binding, React-Native package, or ORT training-web. +- No new WebGPU features — this is a consolidation. --- -## 4. Background: how backend selection works today +## 4. Background: backend selection -Backend choice is a **build-time** decision, not a runtime toggle. It is gated by `BUILD_DEFS` flags baked into -each bundle at build time: - -- `BUILD_DEFS.DISABLE_JSEP` -- `BUILD_DEFS.DISABLE_WEBGPU` (native EP) -- `BUILD_DEFS.DISABLE_WASM` - -The pivot lives in `js/web/script/build.ts` via a temporary `USE_WEBGPU_EP` flag (default `false`). Its -`DEFAULT_DEFINE` sets `DISABLE_JSEP = !!USE_WEBGPU_EP` and `DISABLE_WEBGPU = !USE_WEBGPU_EP`, so a single flag -chooses JSEP vs. the native EP for a given build. Both `USE_WEBGPU_EP` and `USE_JSPI` are already annotated in -the source as temporary and slated for removal. +Whether a bundle uses JSEP or the native WebGPU EP is a build-time choice (distinct from the runtime +`executionProviders` selection), gated by `BUILD_DEFS` flags: `DISABLE_JSEP`, `DISABLE_WEBGPU` (native EP), and +`DISABLE_WASM`. The pivot is the temporary `USE_WEBGPU_EP` flag in `js/web/script/build.ts` (default `false`), +which sets `DISABLE_JSEP = !!USE_WEBGPU_EP` and `DISABLE_WEBGPU = !USE_WEBGPU_EP`. Both `USE_WEBGPU_EP` and +`USE_JSPI` are annotated in the source as temporary. ### 4.1 Current bundle / export map (`js/web/package.json`) @@ -85,198 +70,97 @@ the source as temporary and slated for removal. | `./jspi` | `ort.jspi.*` | Native WebGPU EP + JSPI | | `./wasm` | `ort.wasm.*` | WASM/CPU only | -The `.` (default) and `./all` bundles are the **JSEP** WebGPU path today. `./webgpu` and `./jspi` are already the -**native** EP. WebNN in the default/`all` bundles is **JSEP-hosted**; in `./webgpu` it is native. - -The raw WASM artifacts are variant-specific: `ort-wasm-simd-threaded.jsep.wasm` (JSEP), -`ort-wasm-simd-threaded.asyncify.wasm` (native EP, as used by downstream consumers), `*.jspi.wasm`, and the base -`*.wasm`. +WebNN in the default/`all` bundles is JSEP-hosted; in `./webgpu` it is native. WASM artifacts are variant-specific: +`*.jsep.wasm` (JSEP), `*.asyncify.wasm` (native EP), `*.jspi.wasm`, and base `*.wasm`. --- -## 5. Proposed approach +## 5. Approach -### 5.1 Transparent default swap (JSEP → native EP) +- **Transparent default swap.** Both JSEP and the native EP register under the `webgpu` key, so flipping the + default bundle to the native EP requires no consumer source changes. +- **Escape hatch.** Phase 1 adds a temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`) that pins + JSEP for one release. It is deprecated and warns once, doubling as a parity-bug funnel. +- **`/all` bundle.** `/all` bundles the WebGPU/WebNN backend and WebGL — two independent things changed by two + efforts (this one flips WebGPU/WebNN to native; the WebGL effort drops WebGL). It is kept to avoid breaking + imports. Once both land, `/all` becomes a real alias of the webgpu/default artifact (`.asyncify.wasm`). The two + efforts may land in either order; whichever lands second performs the repoint (§9, WebGL doc §8). -The default `.` bundle already registers its WebGPU backend under the `webgpu` registry key. The native EP also -registers under `webgpu`. Because selection is a build-time swap and the registry key is identical, flipping the -default bundle from JSEP to the native EP is **transparent**: existing code that requests the `webgpu` EP -continues to work with no source changes. - -### 5.2 Escape hatch: temporary `onnxruntime-web/jsep` +--- -After the flip, no existing bundle keeps JSEP (no `/jsep` subpath exists today, and the flip moves both `.` and -`/all` to the native EP). To de-risk, Phase 1 adds a **temporary** `onnxruntime-web/jsep` export (built -`USE_WEBGPU_EP=false`) that pins JSEP for **one release**. It is marked deprecated with a scheduled removal and -emits a one-time warning that doubles as a **parity bug funnel** — surfacing native-EP gaps before JSEP is -deleted. +## 6. int64 handling -### 5.3 `/all` bundle: keep as a converging alias +The native EP reads int64 support from `ep.webgpuexecutionprovider.enableInt64` (`webgpu_provider_factory.cc`), +default `false`. It is reachable on `/webgpu` today via the untyped `extra` map +(`extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }`). -`./all` today bundles two **independent** things: the WebGPU/WebNN backend (JSEP) **and** WebGL. Two independent -efforts change it — this migration flips its WebGPU/WebNN half from JSEP to the native EP, and the -[WebGL removal](onnxruntime_web_remove_webgl_backend.md) drops WebGL. Once **both** have landed, `/all` converges -to **native WebGPU EP + native WebNN**, identical to `./webgpu` and the default `.`. +With int64 **off** (the default), the native EP matches JSEP exactly: int64 arithmetic runs on CPU/WASM with full +`int64_t` precision, while int64 indices (`Gather` / `GatherElements` / `MaxPool`) run on the GPU with i32 +truncation. `enableInt64 = 1` is an opt-in that additionally runs int64 arithmetic on the GPU — faster, but +i32-truncated and therefore lossy for genuine `> 2³¹` values. `transformers.js` already runs the native EP with +int64 off at scale. -**Decision:** keep `/all` to avoid breaking imports. Its **end state** is a **real alias to the same physical -artifact** as the webgpu/default bundle (rather than a separately-built `ort.all.*`) — avoiding bundle drift, -dropping a build target, and not doubling the CDN/WASM payload. The backing WASM shifts `.jsep.wasm` → -`.asyncify.wasm` with the general filename migration. "all" becomes a mild misnomer (no longer implies WebGL) — -documented, not worth an export break. +**Decision:** keep int64 off by default. `enableInt64 = 1` is an opt-in tradeoff, not needed for JSEP parity. +Optional follow-ups: a typed `enableInt64?: boolean` option, and a benchmark of an int64-heavy graph. -**Sequencing (order-independent).** The native flip (this effort) and the WebGL drop (the WebGL effort) are -independent and may land in either order, or in different releases. Until both land, `/all` stays a **distinct -bundle**: after this effort's Phase 1 flip but before WebGL is removed, `/all` is **native WebGPU/WebNN + WebGL** -(still not identical to default/`./webgpu`). Whichever effort lands second collapses `/all` into the physical -alias — see §9 (this effort's Phase 2) and the WebGL doc §8. +Graph capture forces int64 on (`enable_int64_{enable_graph_capture || enable_int64}` in +`webgpu_execution_provider.cc`) to keep the captured region all-GPU; capture users accept i32 truncation, which +fits the LLM-decode token-ID case. --- -## 6. The int64 gap (analysis and decision) - -**Mechanics.** The native EP reads int64 support from the session-config entry -`ep.webgpuexecutionprovider.enableInt64` (`webgpu_provider_factory.cc`); absent, it defaults to `false`. The web -layer has no typed field for it, but it is reachable today on `/webgpu` via the untyped `extra` map -(`iterateExtraOptions` flattens `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }` into the config key). - -**Emulation (verified).** When int64 is **enabled** on GPU, JSEP and the native EP use identical **i32-truncated** -emulation (storage `vec2`, compute `i32`, high word discarded, 32-bit arithmetic). When int64 is **disabled** -(the native default), ops with int64 *inputs* partition to the CPU/WASM fallback using real `int64_t` — full -64-bit correctness. **Exception:** casting *to* int64 stays on GPU even when int64 is off (the WebGPU `Cast` -kernel only guards int64 *inputs*), so "int64 off" narrows to int64 *inputs*, not all int64 work. - -**Why it's not a blocker.** int64 in real models carries indices/shapes/axes/token IDs, all `≪ 2³¹` (a tensor -with `> 2³¹` elements can't fit in a `< 2GB` buffer), so JSEP (GPU-truncated) and native-int64-off (CPU-full) -produce identical numeric results for realistic models — the difference is **performance only**. `transformers.js` -already runs the native EP with int64 **off** at scale on int64-heavy token-ID models, confirming the overhead of -running those int64-input ops on CPU/WASM (plus the partition-boundary copies — int64 inputs typically originate on -CPU, and results are uploaded back to the GPU) is acceptable. - -**Decision.** Keep int64 **off by default** everywhere, with a uniform native-EP config across `.` and `/webgpu`. -Migrating JSEP users (always-truncating) to native-int64-off is a correctness *upgrade*. Follow-ups (not -blockers): (1) an **optional** typed `enableInt64?: boolean` on `WebGpuExecutionProviderOption` as sugar over the -`extra` key that already works today; (2) benchmark an int64-heavy graph (tokenizer / LLM decode) to quantify that -cost (CPU/WASM execution of the int64-input ops plus the partition-boundary copies). - -**Graph-capture interaction (resolved).** Graph capture and int64 are coupled inside the EP -(`enable_int64_{config.enable_graph_capture || config.enable_int64}` in `webgpu_execution_provider.cc`): capture -OFF (default) leaves int64 off (int64 ops on CPU, full correctness); capture ON auto-promotes int64 to keep the -captured region all-GPU/capturable. So the `enableInt64` default never breaks graph capture, and capture users -implicitly accept the i32-truncation regime — exactly the LLM-decode token-ID case (`≪ 2³¹`), the status quo -today. +## 7. Items to validate before the flip ---- +Parity checks to close before Phase 1. Items 1–3 need a real browser + GPU/WebNN run; item 4 is a known wiring gap +to fix. -## 7. Open investigation items (to resolve during implementation) - -Parity concerns to close before the Phase 1 flip. Most are end-to-end validations — the TypeScript wiring is -confirmable by source audit (noted below), but the runtime behavior needs a real browser + GPU/WebNN check. Item -4 is not a runtime unknown but a known, addressable wiring gap — a prerequisite, not a blocker. - -1. **Proxy-worker parity (potential blocker).** `wasm.proxy = true` runs compute in a dedicated worker - (`proxy-wrapper.ts`). *Source:* the wrapper is EP-agnostic and already forbids all GPU I/O over the proxy - (CPU-I/O-only for both backends). *Runtime:* verify the native EP's device init + threading work inside a - Worker under Asyncify. (`transformers.js` forces `proxy = false`, so it is not prior art here.) -2. **IO-binding parity (potential blocker).** *Source:* the `'gpu-buffer'` / `'ml-tensor'` paths in - `wasm-core-impl.ts` have symmetric native/JSEP hooks selected by `BUILD_DEFS.DISABLE_WEBGPU` - (`webgpuRegisterBuffer`/`jsepRegisterBuffer`, etc.). *Runtime:* confirm the native WASM exports deliver true - zero-copy handoff and identical dispose/lifetime semantics. Not covered by op-parity tests. -3. **WebNN native-vs-JSEP parity.** The flip also moves default-bundle WebNN from JSEP-hosted to native — a - second silent swap. *Source:* both init paths are visible in `initEp` (`initJsep('webnn', …)` vs - `webnnInit(...)`). *Runtime:* requires a `navigator.ml`-capable environment; E2E-only. -4. **Global `env.webgpu.*` settings parity (prerequisite).** A known, addressable gap: the default bundle's - global WebGPU knobs are **not** honored by the native EP today. *Source:* `wasm-core-impl.ts` reads - `env.webgpu.adapter` / `powerPreference` / `forceFallbackAdapter` only to call `navigator.gpu.requestAdapter()` - on the JS side, then calls `webgpuInit()` **without passing that adapter** to the native EP — the JSEP branch - does forward it to `initJsep`, so the native path drops it. The C++ factory has a `powerPreference` config key - (`webgpu_provider_factory.cc`) that the TS layer never populates, and `startProfiling()` in - `session-handler-inference.ts` is still a `// TODO` on the native path while JSEP consumes - `env.webgpu.profiling.ondata`. So after the flip, consumers relying on these globals silently lose them. - *Action (required before the flip):* either wire the relevant settings into the native path (or typed - per-session replacements), or explicitly document the dropped/changed settings with migration guidance — a - release-gate item (§8). - -**Closed by source audit — per-session typed-option parity.** The native EP branch in `session-options.ts` honors -a **superset** of JSEP's per-session typed options, so no JSEP-only typed field silently becomes a no-op under -native. JSEP's *global* `env.webgpu.*` configuration is a separate, still-open gap — see item 4. +1. **Proxy-worker (`wasm.proxy = true`).** The proxy wrapper (`proxy-wrapper.ts`) is EP-agnostic and CPU-I/O-only; + verify the native EP's device init and threading work inside a Worker under Asyncify. +2. **IO-binding.** The `'gpu-buffer'` / `'ml-tensor'` paths in `wasm-core-impl.ts` have symmetric native/JSEP + hooks; confirm the native path delivers true zero-copy handoff and matching dispose/lifetime semantics. +3. **WebNN.** The flip also moves default-bundle WebNN from JSEP-hosted to native; validate in a + `navigator.ml`-capable environment. +4. **Global `env.webgpu.*` settings (wiring gap).** `wasm-core-impl.ts` requests the adapter on the JS side but + calls `webgpuInit()` without forwarding it to the native EP, and `startProfiling()` is a TODO on the native + path — so `adapter` / `powerPreference` / `forceFallbackAdapter` / `profiling` are silently dropped. Wire these + into the native path (or document the change) before the flip. + +Per-session typed options are already a superset of JSEP's, confirmed by source audit. --- ## 8. Phase 1 — Flip + deprecate (this release) -Flip the default to the native EP **and** ship the escape hatch, so the swap is protected by a one-release -fallback. Deliverables: - -1. **Flip the default.** Build the default `.` and `./all` bundles with the native EP (`USE_WEBGPU_EP=true`). The - `webgpu` key and public API are unchanged, so the swap is transparent (§5.1). `/all` still bundles WebGL until - the WebGL effort removes it, so it stays a distinct bundle for now (§5.3). Gated on the release gate below. -2. **Escape hatch.** Add the temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`), deprecated - with a scheduled removal, shipping in this same release. -3. **JSEP warn-once.** In `wasm-core-impl.ts` `initEp` under `if (!BUILD_DEFS.DISABLE_JSEP)`, for `epName` - `'webgpu'` (and `'webnn'`). Only the `/jsep` build warns; the native default emits nothing. -4. **Shared deprecation-warning utility.** Warn once and respect `env.logLevel` (error/fatal silences it); no - separate opt-out flag. Shared with the WebGL-removal effort. -5. **Publish the tracking issue ahead of release.** The pinned migration issue (§11) ships **before** this release - as the early-warning channel, substituting for a separate warning-only release. Consumers can stay on the - current pre-flip version until ready, then adopt the flip release and pin the `/jsep` hatch it introduces if - needed. -6. **int64 `extra` opt-in (documented).** Document `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }` for - users needing JSEP-identical GPU int64; works today with no new plumbing, typed field optional (§6). -7. **Docs.** Deprecation banners + migration guidance in `js/web/README` and `js/web/docs/webgpu-operators.md`; a - prominent release-notes/CHANGELOG entry (WASM filename `.jsep.wasm` → `.asyncify.wasm`, `/jsep` pin - instructions). Call out that **WebNN** in `.` / `/all` also moves from JSEP-hosted to native WebNN (the same - path `/webgpu` already ships), so WebNN users know to re-test and use `/jsep` on regression — the - WebGPU-centric framing otherwise hides this second swap (§7.3). - -### Warning placement rationale - -The default `.` bundle (native EP) emits nothing — the implementation swap is not actionable for the consumer and -is communicated via the tracking issue and release notes. The `/jsep` escape-hatch bundle warns once ("deprecated -JSEP build, removed in vX; file an issue if the native WebGPU EP does not work for you"), doubling as a parity bug -funnel. - -### Release gate - -The default flip (#1) is gated on **both**: - -1. **Default-bundle suite runs against the native EP.** CI must exercise the default `.` bundle against the native - EP — a green `/webgpu` run is **not** sufficient, since the bundles differ in wiring (proxy, IO-binding, WebNN - host) even when op kernels match. -2. **§7 blockers resolved with targeted coverage** (op-parity tests don't exercise these paths): the native EP - under `wasm.proxy = true` (§7.1); both `'gpu-buffer'` and `'ml-tensor'` IO-binding zero-copy handoff (§7.2); - native WebNN validated separately (§7.3); and the global `env.webgpu.*` settings gap (§7.4) closed by wiring - `adapter` / `powerPreference` / `forceFallbackAdapter` / `profiling` into the native path (or typed per-session - replacements), or by documenting the dropped/changed settings with migration guidance. Per-session - typed-option parity is already closed by source audit. - -Both gates are required — passing #1 while leaving #2 open would ship acknowledged blocker-class parity gaps -unverified. +1. **Flip the default.** Build `.` and `./all` with the native EP (`USE_WEBGPU_EP=true`). The `webgpu` key and + public API are unchanged. `/all` keeps WebGL until the WebGL effort removes it. +2. **Escape hatch.** Add the temporary, deprecated `onnxruntime-web/jsep` export (`USE_WEBGPU_EP=false`). +3. **Warn once.** The `/jsep` build warns once (respecting `env.logLevel`) via a shared deprecation-warning + utility, also used by the WebGL effort; the native default emits nothing. +4. **Publish the tracking issue** ahead of the release as the early-warning channel. +5. **Docs.** Deprecation banners + migration guidance in `js/web/README` and `js/web/docs/webgpu-operators.md`, + and a release-notes entry covering the `.jsep.wasm` → `.asyncify.wasm` filename change, `/jsep` pinning, the + int64 `extra` opt-in, and the WebNN swap to native (which the WebGPU-centric framing otherwise hides). + +**Release gate.** The flip is gated on both: (a) CI running the default `.` bundle against the native EP — a green +`/webgpu` run is not sufficient, since the bundles differ in proxy/IO-binding/WebNN wiring even when op kernels +match; and (b) the §7 items resolved with targeted coverage (op-parity tests don't exercise those paths). --- ## 9. Phase 2 — Removal (subsequent release) -The default already runs the native EP (flipped in Phase 1); this release deletes JSEP and the temporary surfaces. - -Timed a **single release** after Phase 1 by default, keeping the `/jsep` escape hatch (and thus the JSEP -implementation) available for exactly one release. Extend only if native-EP parity regressions surface in the wild -— real fallback usage via `/jsep`, not a fixed calendar, drives any extension. - -1. **Remove build variants:** drop JSEP WASM artifacts; update `build.ts` and `package.json` exports, and remove - the temporary `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact (§5.3) — valid only once - WebGL has also been dropped from `/all` (WebGL doc §8); if WebGL is still present, `/all` stays a distinct - bundle and the repoint waits on that removal. -2. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. -3. **Relocate WebNN** out of the `jsep/` directory (`backend-webnn.ts`, `webnn/`) to a neutral path. -4. **Remove `pre-jsep.js` glue;** confirm `post-webgpu.js` / `post-webnn.js` cover all initialization. -5. **Remove the temporary `onnxruntime-web/jsep` export.** - -### Removal ergonomics +Timed one release after Phase 1, keeping the `/jsep` hatch available for exactly one release. Extend only if +native-EP parity regressions surface via real `/jsep` usage. -The `/jsep` export is removed cleanly, relying on the native bundler error: a lingering -`import 'onnxruntime-web/jsep'` fails with "subpath ./jsep is not defined by exports" — an acceptable build-time -failure on this temporary, opt-in surface. The default `.` import is unaffected (it already swaps to the native -EP), so no consumer on the documented path hits an error. Communicated via release notes and the tracking issue. +1. Drop JSEP WASM artifacts; update `build.ts` / `package.json`; remove the `USE_WEBGPU_EP` flag. Repoint `/all` + to the webgpu/default artifact — only once WebGL has also been dropped (WebGL doc §8); otherwise `/all` stays a + distinct bundle until then. +2. Delete `BUILD_DEFS.DISABLE_JSEP` and the code it gates; simplify `index.ts`. +3. Relocate WebNN out of `jsep/` (`backend-webnn.ts`, `webnn/`) to a neutral path. +4. Remove `pre-jsep.js` glue; confirm `post-webgpu.js` / `post-webnn.js` cover initialization. +5. Remove the `onnxruntime-web/jsep` export. A lingering `import 'onnxruntime-web/jsep'` then fails with the + native bundler error — an acceptable build-time failure on this temporary surface. The default `.` import is + unaffected. --- @@ -284,36 +168,26 @@ EP), so no consumer on the documented path hits an error. Communicated via relea | Risk | Likelihood | Mitigation | |---|---|---| -| Undiscovered native-EP parity gap vs. JSEP | Medium | Temporary `/jsep` escape hatch (1 release) + warn-once bug funnel; differential tests | -| int64 perf regression (CPU fallback) | Low–Med | int64-off is correctness-preserving; benchmark int64-heavy graph; document `extra`/typed opt-in | -| Proxy-worker path behaves differently | Unknown | **Investigate before flip** (§7.1) | -| IO-binding (gpu-buffer/ml-tensor) semantics differ | Unknown | **Investigate before flip** (§7.2) | -| WebNN behavior changes under native path | Unknown | Validate native WebNN separately (§7.3) | -| Global `env.webgpu.*` settings (adapter/powerPreference/forceFallbackAdapter/profiling) silently dropped on native | Medium | Wire into the native path or document dropped/changed settings with migration guidance — release gate (§7.4, §8) | -| WASM artifact filename change breaks `wasmPaths` | Medium | Document migration; call out `.jsep.wasm` → `.asyncify.wasm` | -| No telemetry on JSEP adoption | Medium | Warn-once funnel + one-release escape hatch as the feedback mechanism | +| Undiscovered native-EP parity gap vs. JSEP | Medium | One-release `/jsep` escape hatch + warn-once funnel; differential tests | +| int64 behavior change vs. JSEP | Low | None by default (native-off matches JSEP); `enableInt64 = 1` is an opt-in tradeoff | +| Proxy / IO-binding / WebNN differ under native | Unknown | Validate before flip (§7) | +| Global `env.webgpu.*` settings dropped on native | Medium | Wire into the native path or document — release gate (§7, §8) | +| WASM filename change breaks `wasmPaths` | Medium | Document `.jsep.wasm` → `.asyncify.wasm` | +| No telemetry on JSEP adoption | Medium | Warn-once funnel + one-release escape hatch | --- ## 11. Migration guide -- **Default import (`onnxruntime-web`):** no code change needed. The `webgpu` backend continues to work; the - implementation swaps to the native EP in Phase 1 (this release). -- **`onnxruntime-web/all`:** continues to work and converges to the native EP. -- **Lower-overhead build (`onnxruntime-web/jspi`):** when you can target JSPI-capable browsers, prefer the `./jspi` - bundle. It runs the **same native WebGPU EP** but bridges async via **JSPI** instead of Asyncify, giving a - smaller WASM binary and lower per-call overhead. The default (Asyncify) build remains the universal fallback for - browsers without JSPI. -- **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). Please file an issue - if the native WebGPU EP does not work for your model so the gap can be fixed before JSEP is deleted. -- **int64-heavy models needing JSEP-identical GPU behavior:** set - `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }` (or the typed option once available). Note this is - lossy for genuine large-integer int64 data; the default (CPU fallback) is more correct. -- **`wasmPaths`:** the backing WASM artifact changes (`.jsep.wasm` → `.asyncify.wasm`); update any hardcoded - paths. - -> **Canonical source:** this effort has its **own** pinned tracking issue (link TBD) as the authoritative, -> continuously-updated migration guidance, independent of the -> [WebGL-removal](onnxruntime_web_remove_webgl_backend.md) issue. Console warnings, README/docs, and CHANGELOG -> entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) links to -> whichever issue is relevant. +- **Default import (`onnxruntime-web`) and `onnxruntime-web/all`:** no code change needed; the `webgpu` backend + keeps working and converges to the native EP. +- **Lower-overhead build (`onnxruntime-web/jspi`):** on JSPI-capable browsers, prefer `./jspi` — the same native + EP with a smaller WASM binary and lower per-call overhead than Asyncify (the universal fallback). +- **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). File an issue if the + native EP does not work for your model. +- **int64-heavy models:** no change needed — the default matches JSEP. To run int64 arithmetic on the GPU (lossy + for genuine `> 2³¹` values), set `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }`. +- **`wasmPaths`:** the backing artifact changes (`.jsep.wasm` → `.asyncify.wasm`); update hardcoded paths. + +> A pinned tracking issue (link TBD) is the authoritative, continuously-updated migration guidance. Warnings, +> docs, and CHANGELOG entries link to it. diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index cb033c4be5c3f..23eff8255311e 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -4,22 +4,22 @@ **Related work:** [Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP](onnxruntime_web_jsep_to_webgpu_ep_migration.md) -— independent, but shares the `onnxruntime-web/all` bundle and the deprecation-warning utility (see §5–§6). +— independent, but shares the `onnxruntime-web/all` bundle (§6) and the deprecation-warning utility (§7). --- ## 1. Summary -`onnxruntime-web` ships a legacy **WebGL** backend — the `onnxjs` implementation under `js/web/lib/onnxjs`, -registered under the `'webgl'` backend key. It predates the current WASM architecture, supports a narrower -operator set, and has fp32/behavioral drift relative to the WebGPU path. +`onnxruntime-web` ships a legacy **WebGL** backend (the `onnxjs` implementation under `js/web/lib/onnxjs`, +registered under the `'webgl'` key). It predates the WASM architecture, supports fewer operators, and has fp32/op +drift relative to WebGPU. -This document proposes **deprecating and then removing** it as a straightforward **deprecate-and-delete** with an -explicit migration message — no transparent redirect (see §5.1), unlike the JSEP → native WebGPU EP migration: +This document proposes deprecating and then deleting it, with an explicit migration message rather than a +transparent redirect (§5.1): - **Phase 1 (this release):** deprecation warning + docs. No behavior change. -- **Phase 2 (subsequent release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` build - variant. +- **Phase 2 (subsequent release):** delete the `onnxjs` backend, its `'webgl'` registration, and the `ort.webgl` + build variant. --- @@ -52,40 +52,34 @@ explicit migration message — no transparent redirect (see §5.1), unlike the J ## 4. Background -The WebGL backend is the `onnxjs` implementation in `js/web/lib/onnxjs`, exposed via `backend-onnxjs.ts` and -registered under the `'webgl'` backend key with **negative priority** — meaning it is excluded from the default -fallback ordering and only used when explicitly requested (`executionProviders: ['webgl']`). - -It appears in two bundles: +The WebGL backend (`onnxjs`, exposed via `backend-onnxjs.ts`) is registered under the `'webgl'` key with +**negative priority**, so it is excluded from the default fallback and used only when explicitly requested +(`executionProviders: ['webgl']`). It appears in two bundles: | Export | Artifact | Contents | |---|---|---| | `./webgl` | `ort.webgl.*` | WebGL only (`DISABLE_WASM: true`) | | `./all` | `ort.all.*` | JSEP WebGPU + WebNN **+ WebGL** | -The `./webgl` bundle sets `BUILD_DEFS.DISABLE_WASM: true`, so it has **no WASM/CPU fallback** — it is WebGL or -nothing. +The `./webgl` bundle sets `DISABLE_WASM: true`, so it has no WASM/CPU fallback — it is WebGL or nothing. ### 4.1 WebGPU browser coverage -The historical reason to keep WebGL was "WebGPU isn't available on Safari / iOS / Firefox." As of 2026 that is -**less true** — WebGPU has shipped enabled-by-default in Chrome/Edge since **113** (2023), in Safari (macOS/iOS), -and in Chrome for Android; Firefox support is partial. Per MDN's -[`api.GPU`](https://developer.mozilla.org/docs/Web/API/GPU#browser_compatibility) browser-compat data, gaps -remain: +WebGL was historically kept for platforms without WebGPU. As of 2026 that is less true: WebGPU has shipped +enabled-by-default in Chrome/Edge (since **113**, 2023), Safari (macOS/iOS), and Chrome for Android; Firefox +support is partial. Per +[MDN `api.GPU`](https://developer.mozilla.org/docs/Web/API/GPU#browser_compatibility): | Browser | WebGPU | |---|---| -| Chrome / Edge (desktop) | ✅ (default since 113; Linux support landed later) | +| Chrome / Edge (desktop) | ✅ (default since 113; Linux later) | | Chrome (Android) | ✅ | | Safari (macOS / iOS) | ✅ | | Firefox (desktop) | ⚠️ partial | | Firefox (Android) | ❌ | -Chrome/Edge and Safari/iOS are solid, but gaps remain (older versions, Firefox for Android, Firefox desktop still -partial). Coverage is **broadening but not universal**, so the pool of users for whom WebGL is the *only* GPU path -is shrinking, not gone. The design does **not** assume universal WebGPU — the actionable fallback for uncovered -users is the WASM/CPU backend. +Coverage is broadening but not universal, so this design does not assume universal WebGPU — the fallback for +uncovered users is the WASM/CPU backend. --- @@ -93,71 +87,58 @@ users is the WASM/CPU backend. ### 5.1 Explicit removal -WebGL cannot be transparently redirected (unlike the WebGPU JSEP → native swap): - -- The `./webgl` bundle sets `DISABLE_WASM: true`, so there is **no WASM/CPU fallback** to redirect to. -- WebGPU requires `navigator.gpu`; a silent redirect would fail where WebGPU is unavailable. -- WebGL's fp32/op drift means silently swapping results would be a hidden behavioral change. - -So WebGL gets an **explicit deprecation warning + removal** directing users to the WebGPU EP (`webgpu`) or the -WASM/CPU backend (`wasm`). +WebGL cannot be transparently redirected: the `./webgl` bundle has no WASM/CPU fallback (`DISABLE_WASM: true`), a +silent WebGPU redirect would fail where `navigator.gpu` is unavailable, and WebGL's fp32/op drift would make a +silent swap a hidden behavioral change. So it gets an explicit deprecation warning + removal directing users to +the WebGPU EP (`webgpu`) or the WASM/CPU backend (`wasm`). ### 5.2 Warning placement -The warning fires in `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), in `init()` / -`createInferenceSessionHandler`. Because the backend has negative priority, it only fires when WebGL is -**explicitly requested** — no spam for consumers who never opt in. Fires for both `ort.all` and `ort.webgl`. +The warning fires in `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), in `init(backendName)` / +`createInferenceSessionHandler`. Negative priority means it only fires when WebGL is explicitly requested — for +both `ort.all` and `ort.webgl`. ### 5.3 Removal ergonomics -Phase 2 relies on the native failure modes rather than adding tombstone stubs or a `'webgl'`-key shim: - -- A lingering `import 'onnxruntime-web/webgl'` fails with the native bundler error ("subpath ./webgl is not - defined by exports") — a clear build-time failure. -- `executionProviders: ['webgl']` throws "no available backend found" from `resolveBackendAndExecutionProviders`. - -WebGL was never a first-class backend (narrow op set, fp32 drift, negative priority), so these built-in errors are -a sufficient soft-landing. +Phase 2 relies on native failure modes rather than tombstone stubs: a lingering `import 'onnxruntime-web/webgl'` +fails with the bundler error ("subpath ./webgl is not defined by exports"), and `executionProviders: ['webgl']` +throws "no available backend found". Given WebGL's narrow op set and negative priority, these built-in errors are +a sufficient soft landing. --- ## 6. `/all` bundle coupling -`./all` bundles two **independent** things: WebGL **and** the WebGPU/WebNN backend. This effort owns **removing -WebGL from `/all`**; the [JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) owns flipping -the WebGPU/WebNN half from JSEP to the native EP. The two changes are independent and may land in either order, or -in different releases; `/all` collapses into a single physical alias of the webgpu/default bundle only once -**both** have landed (whichever lands second performs the repoint). `/all` is **kept** as an export to avoid -breaking imports (see JSEP doc §5.3 for the alias decision). +`./all` bundles two independent things: WebGL and the WebGPU/WebNN backend. This effort removes WebGL; the +[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) flips WebGPU/WebNN to the native EP. The +two are independent and may land in either order; `/all` collapses into a single alias of the webgpu/default +bundle once both have landed (whichever lands second performs the repoint). `/all` is kept as an export to avoid +breaking imports (see JSEP doc §5). --- ## 7. Phase 1 — Deprecation (this release) -No behavior change. Deliverables: +No behavior change. -1. **WebGL warn-once.** In `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), `init()` / - `createInferenceSessionHandler`. Gate on `logLevel`; dedupe via a module flag. Uses the shared - deprecation-warning utility (built by the JSEP effort, or here — whichever lands first). -2. **Docs.** Deprecation banner + migration guidance in `js/web/README` and `js/web/docs/webgl-operators.md`; - release notes / CHANGELOG entries. +1. **WebGL warn-once.** In `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), `init(backendName)` / + `createInferenceSessionHandler`, gated on `logLevel` and deduped via a module flag. Uses the shared + deprecation-warning utility (from the JSEP effort, or here — whichever lands first). +2. **Docs.** Deprecation banner + migration guidance in `js/web/README` and `js/web/docs/webgl-operators.md`, plus + release notes. --- ## 8. Phase 2 — Removal (subsequent release) -Timed a **single release** after Phase 1 by default, kept **independent** of the JSEP removal's schedule rather -than forcing the two to align. Extend only if issues surface during deprecation (e.g. WebGL-reliant consumers -report friction migrating to `wasm`/WebGPU) — real-world feedback, not the JSEP timeline, drives any extension. +Timed one release after Phase 1, kept independent of the JSEP removal schedule. Extend only if WebGL-reliant +consumers report friction migrating to `wasm`/WebGPU. -1. **Delete WebGL:** remove the `onnxjs` backend (`js/web/lib/onnxjs`), the `'webgl'` registration, and - `backend-onnxjs.ts`. -2. **Remove the `ort.webgl` build variant;** update `build.ts` and `package.json` exports (remove the `./webgl` - export). -3. **Drop WebGL from `ort.all`** (see §6). If the JSEP → native flip has already landed, this is the step that - collapses `/all` into the webgpu/default alias; if not, `/all` remains a distinct JSEP WebGPU/WebNN bundle - until that flip lands. -4. **Remove guarded code:** delete `BUILD_DEFS.DISABLE_WEBGL` and the code it gates; simplify `index.ts`. +1. Delete the `onnxjs` backend (`js/web/lib/onnxjs`), the `'webgl'` registration, and `backend-onnxjs.ts`. +2. Remove the `ort.webgl` build variant and the `./webgl` export. +3. Drop WebGL from `ort.all` (§6). If the JSEP → native flip has landed, this collapses `/all` into the + webgpu/default alias; otherwise `/all` stays a distinct JSEP bundle until it does. +4. Delete `BUILD_DEFS.DISABLE_WEBGL` and the code it gates; simplify `index.ts`. --- @@ -165,9 +146,9 @@ report friction migrating to `wasm`/WebGPU) — real-world feedback, not the JSE | Risk | Likelihood | Mitigation | |---|---|---| -| Consumers relying on WebGL as a fallback where WebGPU is unavailable | Medium | WebGPU coverage is broadening but still has platform caveats (§4.1); the actionable fallback for uncovered users is `wasm`, not WebGPU; explicit migration message + deprecation window + release notes | -| Silent breakage from `./webgl` import removal | Low | Deprecate first; document removal release; no transparent redirect (avoids hidden drift) | -| No telemetry on WebGL adoption | Medium | Warn-once as the feedback mechanism during the deprecation window | +| Consumers relying on WebGL where WebGPU is unavailable | Medium | Coverage is broadening but not universal (§4.1); the fallback is `wasm`; explicit migration message + deprecation window | +| Silent breakage from `./webgl` import removal | Low | Deprecate first; document the removal release; no transparent redirect | +| No telemetry on WebGL adoption | Medium | Warn-once as the feedback mechanism during deprecation | --- @@ -179,8 +160,5 @@ report friction migrating to `wasm`/WebGPU) — real-world feedback, not the JSE - **`onnxruntime-web/all`:** continues to work, but no longer includes WebGL. If you relied on WebGL as a fallback via `/all`, add `wasm` to your `executionProviders` list. -> **Canonical source:** this effort has its **own** pinned tracking issue (link TBD) as the authoritative, -> continuously-updated migration guidance, independent of the -> [JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) issue. Console warnings, README/docs, -> and CHANGELOG entries **link** to it rather than duplicating guidance. `/all` guidance (touched by both efforts) -> links to whichever issue is relevant. +> A pinned tracking issue (link TBD) is the authoritative migration guidance. Warnings, docs, and CHANGELOG +> entries link to it. From 89f4939d97369ebdf23c17e202fc4cac1effa124 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Fri, 31 Jul 2026 15:42:36 -0700 Subject: [PATCH 08/10] Address PR review comments on onnxruntime-web backend deprecation design docs Refine the JSEP-to-native-WebGPU-EP migration doc and the WebGL removal doc based on PR review feedback: correct the build-config/CI-coverage claims (the default and ./webgpu bundles are the same build; proxy, IO-binding, and WebNN are unexercised against the native EP in CI rather than differing), fix the WASM artifact suffix mapping, and tighten the validation items and release gate. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 114 ++++++++++-------- .../onnxruntime_web_remove_webgl_backend.md | 15 ++- 2 files changed, 75 insertions(+), 54 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index 0d8a5918bf735..6fae23a985ce3 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -11,15 +11,16 @@ `onnxruntime-web` implements WebGPU/WebNN compute two ways: -- **JSEP** — WebGPU/WebNN implemented in TypeScript over an Asyncify-compiled WASM core. Powers the default (`.`) - and `./all` bundles. -- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM. Powers the `./webgpu` and `./jspi` - bundles. +- **JSEP** — the WebGPU compute path implemented in TypeScript over an Asyncify-compiled WASM core (it also hosts + WebNN). Powers the default (`.`) and `./all` bundles. +- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM (hosting WebNN natively). Powers the + `./webgpu` and `./jspi` bundles. This document proposes removing JSEP and standardizing on the native WebGPU EP. The default bundle keeps the -`webgpu` backend key and swaps implementation at build time, so the change is transparent for existing code. +`webgpu` backend key and swaps implementation at build time, so the change is transparent for existing code. WebNN +comes along unchanged — the default bundle just switches to the native WebNN host that `./webgpu` already uses. -- **Phase 1 (this release):** flip the default (`.`) and `./all` bundles to the native EP, add a temporary +- **Phase 1 (this release):** flip the default (`.`) and `./all` bundles to the native WebGPU EP, add a temporary `onnxruntime-web/jsep` escape-hatch export for one release, and ship deprecation warnings + docs. - **Phase 2 (subsequent release):** delete JSEP and remove the temporary `/jsep` export and build flags. @@ -27,8 +28,8 @@ This document proposes removing JSEP and standardizing on the native WebGPU EP. ## 2. Motivation -- **Remove duplicate maintenance.** JSEP and the native EP implement the same operators twice (TypeScript and - C++). The native EP is the strategic direction and receives new op work. +- **Remove duplicate maintenance.** JSEP and the native WebGPU EP implement the same operators twice (TypeScript + and C++). The native WebGPU EP is the strategic direction and receives new op work. - **Consistent runtime semantics.** The browser build shares kernel behavior with the rest of ONNX Runtime instead of maintaining a parallel TS implementation. - **Simpler build matrix.** Removing JSEP deletes the temporary `USE_WEBGPU_EP` / `DISABLE_JSEP` build plumbing. @@ -55,30 +56,27 @@ This document proposes removing JSEP and standardizing on the native WebGPU EP. ## 4. Background: backend selection Whether a bundle uses JSEP or the native WebGPU EP is a build-time choice (distinct from the runtime -`executionProviders` selection), gated by `BUILD_DEFS` flags: `DISABLE_JSEP`, `DISABLE_WEBGPU` (native EP), and -`DISABLE_WASM`. The pivot is the temporary `USE_WEBGPU_EP` flag in `js/web/script/build.ts` (default `false`), +`executionProviders` selection), gated by the `BUILD_DEFS` flags `DISABLE_JSEP` and `DISABLE_WEBGPU` (native WebGPU +EP). The pivot is the temporary `USE_WEBGPU_EP` flag in `js/web/script/build.ts` (default `false`), which sets `DISABLE_JSEP = !!USE_WEBGPU_EP` and `DISABLE_WEBGPU = !USE_WEBGPU_EP`. Both `USE_WEBGPU_EP` and `USE_JSPI` are annotated in the source as temporary. ### 4.1 Current bundle / export map (`js/web/package.json`) -| Export | Artifact | WebGPU/WebNN path | -|---|---|---| -| `.` (default) | `ort.min` / `ort.bundle.min` | JSEP WebGPU + WebNN | -| `./all` | `ort.all.*` | JSEP WebGPU + WebNN (+ WebGL) | -| `./webgpu` | `ort.webgpu.*` | **Native** WebGPU EP + native WebNN | -| `./jspi` | `ort.jspi.*` | Native WebGPU EP + JSPI | -| `./wasm` | `ort.wasm.*` | WASM/CPU only | - -WebNN in the default/`all` bundles is JSEP-hosted; in `./webgpu` it is native. WASM artifacts are variant-specific: -`*.jsep.wasm` (JSEP), `*.asyncify.wasm` (native EP), `*.jspi.wasm`, and base `*.wasm`. +| Export | WebGPU impl | Async bridge | WASM binary suffix | +|---|---|---|---| +| `.` (default) | JSEP | Asyncify | `.jsep.wasm` | +| `./all` | JSEP | Asyncify | `.jsep.wasm` | +| `./webgpu` | native WebGPU EP | Asyncify | `.asyncify.wasm` | +| `./jspi` | native WebGPU EP | JSPI | `.jspi.wasm` | +| `./wasm` | — (CPU only) | — | `.wasm` | --- ## 5. Approach -- **Transparent default swap.** Both JSEP and the native EP register under the `webgpu` key, so flipping the - default bundle to the native EP requires no consumer source changes. +- **Transparent default swap.** Both JSEP and the native WebGPU EP register under the `webgpu` key, so flipping + the default bundle to the native WebGPU EP requires no consumer source changes. - **Escape hatch.** Phase 1 adds a temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`) that pins JSEP for one release. It is deprecated and warns once, doubling as a parity-bug funnel. - **`/all` bundle.** `/all` bundles the WebGPU/WebNN backend and WebGL — two independent things changed by two @@ -90,22 +88,23 @@ WebNN in the default/`all` bundles is JSEP-hosted; in `./webgpu` it is native. W ## 6. int64 handling -The native EP reads int64 support from `ep.webgpuexecutionprovider.enableInt64` (`webgpu_provider_factory.cc`), +The native WebGPU EP reads int64 support from `ep.webgpuexecutionprovider.enableInt64` (`webgpu_provider_factory.cc`), default `false`. It is reachable on `/webgpu` today via the untyped `extra` map (`extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }`). -With int64 **off** (the default), the native EP matches JSEP exactly: int64 arithmetic runs on CPU/WASM with full +With int64 **off** (the default), the native WebGPU EP matches JSEP exactly: int64 arithmetic runs on CPU/WASM with full `int64_t` precision, while int64 indices (`Gather` / `GatherElements` / `MaxPool`) run on the GPU with i32 truncation. `enableInt64 = 1` is an opt-in that additionally runs int64 arithmetic on the GPU — faster, but -i32-truncated and therefore lossy for genuine `> 2³¹` values. `transformers.js` already runs the native EP with -int64 off at scale. +i32-truncated and therefore lossy for genuine `> 2³¹` values. `transformers.js` already runs the native WebGPU EP +with int64 off at scale. **Decision:** keep int64 off by default. `enableInt64 = 1` is an opt-in tradeoff, not needed for JSEP parity. -Optional follow-ups: a typed `enableInt64?: boolean` option, and a benchmark of an int64-heavy graph. +Optional follow-up: a typed `enableInt64?: boolean` option. Graph capture forces int64 on (`enable_int64_{enable_graph_capture || enable_int64}` in -`webgpu_execution_provider.cc`) to keep the captured region all-GPU; capture users accept i32 truncation, which -fits the LLM-decode token-ID case. +`webgpu_execution_provider.cc`) to keep the captured region all-GPU. So `enableGraphCapture = true` silently moves +int64 arithmetic to the GPU and truncates genuine `> 2³¹` values — an exception to default parity, flagged in the +migration guide (§11). --- @@ -114,14 +113,18 @@ fits the LLM-decode token-ID case. Parity checks to close before Phase 1. Items 1–3 need a real browser + GPU/WebNN run; item 4 is a known wiring gap to fix. -1. **Proxy-worker (`wasm.proxy = true`).** The proxy wrapper (`proxy-wrapper.ts`) is EP-agnostic and CPU-I/O-only; - verify the native EP's device init and threading work inside a Worker under Asyncify. +1. **Proxy-worker (`wasm.proxy = true`).** `./webgpu` already ships this exact path (native EP + Asyncify + proxy + over the EP-agnostic `proxy-wrapper.ts`), so this is a coverage check, not new wiring. CI today runs `--wasm.proxy` + only with `-b=wasm` (CPU) on the JSEP build — never `-b=webgpu`, and the `--webgpu-ep` leg runs no proxy. Confirm a + real proxy-mode run on a GPU: native EP device init and threading inside a Worker. 2. **IO-binding.** The `'gpu-buffer'` / `'ml-tensor'` paths in `wasm-core-impl.ts` have symmetric native/JSEP - hooks; confirm the native path delivers true zero-copy handoff and matching dispose/lifetime semantics. -3. **WebNN.** The flip also moves default-bundle WebNN from JSEP-hosted to native; validate in a - `navigator.ml`-capable environment. + hooks. CI today runs the `--io-binding=gpu-tensor` / `gpu-location` legs on the JSEP build only; the `--webgpu-ep` + leg passes no `--io-binding`. Confirm the native path delivers true zero-copy handoff and matching dispose/lifetime + semantics on a GPU. +3. **WebNN.** The default bundle switches to the same native WebNN host `./webgpu` already ships, so WebNN itself + is unchanged; just confirm the default bundle behaves the same in a `navigator.ml`-capable environment. 4. **Global `env.webgpu.*` settings (wiring gap).** `wasm-core-impl.ts` requests the adapter on the JS side but - calls `webgpuInit()` without forwarding it to the native EP, and `startProfiling()` is a TODO on the native + calls `webgpuInit()` without forwarding it to the native WebGPU EP, and `startProfiling()` is a TODO on the native path — so `adapter` / `powerPreference` / `forceFallbackAdapter` / `profiling` are silently dropped. Wire these into the native path (or document the change) before the flip. @@ -131,19 +134,25 @@ Per-session typed options are already a superset of JSEP's, confirmed by source ## 8. Phase 1 — Flip + deprecate (this release) -1. **Flip the default.** Build `.` and `./all` with the native EP (`USE_WEBGPU_EP=true`). The `webgpu` key and +1. **Flip the default.** Build `.` and `./all` with the native EPs (`USE_WEBGPU_EP=true`). The `webgpu` key and public API are unchanged. `/all` keeps WebGL until the WebGL effort removes it. 2. **Escape hatch.** Add the temporary, deprecated `onnxruntime-web/jsep` export (`USE_WEBGPU_EP=false`). 3. **Warn once.** The `/jsep` build warns once (respecting `env.logLevel`) via a shared deprecation-warning utility, also used by the WebGL effort; the native default emits nothing. 4. **Publish the tracking issue** ahead of the release as the early-warning channel. -5. **Docs.** Deprecation banners + migration guidance in `js/web/README` and `js/web/docs/webgpu-operators.md`, - and a release-notes entry covering the `.jsep.wasm` → `.asyncify.wasm` filename change, `/jsep` pinning, the - int64 `extra` opt-in, and the WebNN swap to native (which the WebGPU-centric framing otherwise hides). - -**Release gate.** The flip is gated on both: (a) CI running the default `.` bundle against the native EP — a green -`/webgpu` run is not sufficient, since the bundles differ in proxy/IO-binding/WebNN wiring even when op kernels -match; and (b) the §7 items resolved with targeted coverage (op-parity tests don't exercise those paths). +5. **Docs.** Deprecation banners + migration guidance in `js/web/README` (hand-authored) and a release-notes entry + covering the `.jsep.wasm` → `.asyncify.wasm` filename change, `/jsep` pinning, the int64 `extra` opt-in, and the + default bundle's WebNN switching to the native host (already used by `./webgpu`). `webgpu-operators.md` is + generated — emit any banner from its generator (item 6), not by hand. +6. **Retarget the operator-doc generator.** `generate-webgpu-operator-md.ts` parses the JSEP registrations + (`js_execution_provider.cc`, `js_contrib_kernels.cc`), so post-flip `webgpu-operators.md` describes the wrong + EP; point it at the native WebGPU EP registrations (those JSEP sources are removed in Phase 2). + +**Release gate.** The flip is gated on both: (a) a **blocking** CI job running the native WebGPU EP — the current +native-WebGPU-EP legs are advisory (`continue-on-error` in `windows-web-ci-workflow.yml`, `continueOnError` in +`win-web-ci.yml`) and must be promoted to blocking. The default `.` bundle and `./webgpu` are the same build, so a +green `/webgpu` run covers the flip only if it exercises proxy/IO-binding/WebNN — which the current op-parity leg +does not; and (b) the §7 items resolved with targeted coverage (op-parity tests don't exercise those paths). --- @@ -170,7 +179,7 @@ native-EP parity regressions surface via real `/jsep` usage. |---|---|---| | Undiscovered native-EP parity gap vs. JSEP | Medium | One-release `/jsep` escape hatch + warn-once funnel; differential tests | | int64 behavior change vs. JSEP | Low | None by default (native-off matches JSEP); `enableInt64 = 1` is an opt-in tradeoff | -| Proxy / IO-binding / WebNN differ under native | Unknown | Validate before flip (§7) | +| Proxy / IO-binding / WebNN unexercised under the native EP in CI | Unknown | Validate before flip (§7) | | Global `env.webgpu.*` settings dropped on native | Medium | Wire into the native path or document — release gate (§7, §8) | | WASM filename change breaks `wasmPaths` | Medium | Document `.jsep.wasm` → `.asyncify.wasm` | | No telemetry on JSEP adoption | Medium | Warn-once funnel + one-release escape hatch | @@ -179,15 +188,18 @@ native-EP parity regressions surface via real `/jsep` usage. ## 11. Migration guide -- **Default import (`onnxruntime-web`) and `onnxruntime-web/all`:** no code change needed; the `webgpu` backend - keeps working and converges to the native EP. +- **Default import (`onnxruntime-web`) and `onnxruntime-web/all`:** no source change if you use default, + package-managed asset resolution; the `webgpu` backend keeps working and converges to the native WebGPU EP. + Consumers that pin WASM artifacts must still update the path (see `wasmPaths` below). - **Lower-overhead build (`onnxruntime-web/jspi`):** on JSPI-capable browsers, prefer `./jspi` — the same native - EP with a smaller WASM binary and lower per-call overhead than Asyncify (the universal fallback). + WebGPU EP with a smaller WASM binary and lower per-call overhead than Asyncify (the universal fallback). - **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). File an issue if the - native EP does not work for your model. -- **int64-heavy models:** no change needed — the default matches JSEP. To run int64 arithmetic on the GPU (lossy - for genuine `> 2³¹` values), set `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }`. -- **`wasmPaths`:** the backing artifact changes (`.jsep.wasm` → `.asyncify.wasm`); update hardcoded paths. + native EPs do not work for your model. +- **int64-heavy models:** no change needed — the default matches JSEP. Exceptions that move int64 arithmetic to the + GPU (lossy for genuine `> 2³¹` values): `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }`, and + `enableGraphCapture = true` (forces int64 on; see §6). +- **`wasmPaths` / pinned artifacts:** the backing artifact changes (`.jsep.wasm` → `.asyncify.wasm`). Update + object-form `env.wasm.wasmPaths`, direct artifact imports, preload/CSP rules, and any copied assets. > A pinned tracking issue (link TBD) is the authoritative, continuously-updated migration guidance. Warnings, > docs, and CHANGELOG entries link to it. diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index 23eff8255311e..6b2d703ada2d2 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -124,8 +124,7 @@ No behavior change. 1. **WebGL warn-once.** In `OnnxjsBackend` (`js/web/lib/backend-onnxjs.ts`), `init(backendName)` / `createInferenceSessionHandler`, gated on `logLevel` and deduped via a module flag. Uses the shared deprecation-warning utility (from the JSEP effort, or here — whichever lands first). -2. **Docs.** Deprecation banner + migration guidance in `js/web/README` and `js/web/docs/webgl-operators.md`, plus - release notes. +2. **Docs.** Deprecation banner + migration guidance in `js/web/README` (hand-authored), plus release notes. --- @@ -134,7 +133,17 @@ No behavior change. Timed one release after Phase 1, kept independent of the JSEP removal schedule. Extend only if WebGL-reliant consumers report friction migrating to `wasm`/WebGPU. -1. Delete the `onnxjs` backend (`js/web/lib/onnxjs`), the `'webgl'` registration, and `backend-onnxjs.ts`. +**Versioning.** ONNX Runtime follows SemVer for its stable public API ([docs/Versioning.md](../Versioning.md)), +but WebGL has always been experimental/maintenance-mode — negative priority, partial op coverage, never GA — so +removing it in a minor release after a deprecation window is within policy and does not require a major bump. + +1. Delete the whole `onnxjs` tree, but not with a naive `rm -rf`. Production reaches it only through + `backend-onnxjs.ts` and the `'webgl'` registration, so remove those plus the WebGL-only code (`onnxjs/backends/webgl`, + the `test/unittests/backends/webgl/*` tests, and `generate-webgl-operator-md.ts`) outright. First relocate the + few shared, non-WebGL utilities that test infra imports — protobuf/ONNX decoding (`ort-schema/protobuf`, + `tensor`), `instrument` (`Logger`/`Profiler`), and `util` (`ProtoUtil`, `PoolConvUtil`, `ShapeUtil`) — to a + neutral path and repoint their importers; then delete the legacy `onnxjs` session, graph, execution engine, and + operators (git history preserves them). 2. Remove the `ort.webgl` build variant and the `./webgl` export. 3. Drop WebGL from `ort.all` (§6). If the JSEP → native flip has landed, this collapses `/all` into the webgpu/default alias; otherwise `/all` stays a distinct JSEP bundle until it does. From c8c83da759d2194f256d22c8c09e9eac3e670b54 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Fri, 31 Jul 2026 17:39:34 -0700 Subject: [PATCH 09/10] Clarify WebNN framing in onnxruntime-web deprecation design docs Address further PR review feedback: WebNN is already the native C++ WebNN EP in both the JSEP and native-WebGPU builds, so the migration only removes JSEP's shared init glue (jsepInit -> webnnInit, same WebNNBackend) rather than moving WebNN to a native host. Restrict the native-WebGPU-EP migration language to WebGPU throughout, and drop the deprecation banner from the generated webgpu-operators.md (guidance stays in the hand-authored README). Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- ...runtime_web_jsep_to_webgpu_ep_migration.md | 31 ++++++++++--------- .../onnxruntime_web_remove_webgl_backend.md | 4 +-- 2 files changed, 18 insertions(+), 17 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index 6fae23a985ce3..ffd669ccc57c0 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -1,6 +1,6 @@ -# Design: Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP +# Design: Migrate onnxruntime-web from JSEP to the native WebGPU EP -**Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU and WebNN backends +**Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGPU backend; WebNN initialization glue **Related work:** [Remove the WebGL (onnxjs) backend from onnxruntime-web](onnxruntime_web_remove_webgl_backend.md) — independent, but shares the `onnxruntime-web/all` bundle and the deprecation-warning utility. @@ -9,16 +9,18 @@ ## 1. Summary -`onnxruntime-web` implements WebGPU/WebNN compute two ways: +`onnxruntime-web` implements WebGPU compute two ways: -- **JSEP** — the WebGPU compute path implemented in TypeScript over an Asyncify-compiled WASM core (it also hosts - WebNN). Powers the default (`.`) and `./all` bundles. -- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM (hosting WebNN natively). Powers the - `./webgpu` and `./jspi` bundles. +- **JSEP** — the WebGPU compute path implemented in TypeScript over an Asyncify-compiled WASM core. Powers the + default (`.`) and `./all` bundles. +- **Native WebGPU EP** — the C++ WebGPU execution provider compiled to WASM. Powers the `./webgpu` and `./jspi` + bundles. This document proposes removing JSEP and standardizing on the native WebGPU EP. The default bundle keeps the `webgpu` backend key and swaps implementation at build time, so the change is transparent for existing code. WebNN -comes along unchanged — the default bundle just switches to the native WebNN host that `./webgpu` already uses. +is unaffected: it is already the native C++ WebNN EP in both builds (both link `--use_webnn`; `session-options.ts` +selects the `WEBNN` EP either way). Removing JSEP only drops the shared JS init glue (`jsepInit('webnn', …)` → +`webnnInit(…)`, same `WebNNBackend`), not the WebNN EP itself. - **Phase 1 (this release):** flip the default (`.`) and `./all` bundles to the native WebGPU EP, add a temporary `onnxruntime-web/jsep` escape-hatch export for one release, and ship deprecation warnings + docs. @@ -40,7 +42,7 @@ comes along unchanged — the default bundle just switches to the native WebNN h ### Goals -- Make the native WebGPU EP the default WebGPU/WebNN backend, transparently for existing consumers (same import, +- Make the native WebGPU EP the default WebGPU backend, transparently for existing consumers (same import, `webgpu` key, and public API). - Give JSEP consumers a low-effort migration path plus a one-release safety net. - Remove the JSEP TypeScript compute path and its build variants. @@ -80,7 +82,7 @@ which sets `DISABLE_JSEP = !!USE_WEBGPU_EP` and `DISABLE_WEBGPU = !USE_WEBGPU_EP - **Escape hatch.** Phase 1 adds a temporary `onnxruntime-web/jsep` export (built `USE_WEBGPU_EP=false`) that pins JSEP for one release. It is deprecated and warns once, doubling as a parity-bug funnel. - **`/all` bundle.** `/all` bundles the WebGPU/WebNN backend and WebGL — two independent things changed by two - efforts (this one flips WebGPU/WebNN to native; the WebGL effort drops WebGL). It is kept to avoid breaking + efforts (this one flips WebGPU to native; the WebGL effort drops WebGL). It is kept to avoid breaking imports. Once both land, `/all` becomes a real alias of the webgpu/default artifact (`.asyncify.wasm`). The two efforts may land in either order; whichever lands second performs the repoint (§9, WebGL doc §8). @@ -121,8 +123,9 @@ to fix. hooks. CI today runs the `--io-binding=gpu-tensor` / `gpu-location` legs on the JSEP build only; the `--webgpu-ep` leg passes no `--io-binding`. Confirm the native path delivers true zero-copy handoff and matching dispose/lifetime semantics on a GPU. -3. **WebNN.** The default bundle switches to the same native WebNN host `./webgpu` already ships, so WebNN itself - is unchanged; just confirm the default bundle behaves the same in a `navigator.ml`-capable environment. +3. **WebNN.** WebNN is already the native C++ WebNN EP in both builds (`session-options.ts` selects `WEBNN` + either way); the flip only swaps the JS init bridge (`jsepInit('webnn', …)` → `webnnInit(…)`, same + `WebNNBackend`). Confirm the native `webnnInit` path behaves the same in a `navigator.ml`-capable environment. 4. **Global `env.webgpu.*` settings (wiring gap).** `wasm-core-impl.ts` requests the adapter on the JS side but calls `webgpuInit()` without forwarding it to the native WebGPU EP, and `startProfiling()` is a TODO on the native path — so `adapter` / `powerPreference` / `forceFallbackAdapter` / `profiling` are silently dropped. Wire these @@ -141,9 +144,7 @@ Per-session typed options are already a superset of JSEP's, confirmed by source utility, also used by the WebGL effort; the native default emits nothing. 4. **Publish the tracking issue** ahead of the release as the early-warning channel. 5. **Docs.** Deprecation banners + migration guidance in `js/web/README` (hand-authored) and a release-notes entry - covering the `.jsep.wasm` → `.asyncify.wasm` filename change, `/jsep` pinning, the int64 `extra` opt-in, and the - default bundle's WebNN switching to the native host (already used by `./webgpu`). `webgpu-operators.md` is - generated — emit any banner from its generator (item 6), not by hand. + covering the `.jsep.wasm` → `.asyncify.wasm` filename change, `/jsep` pinning, and the int64 `extra` opt-in. 6. **Retarget the operator-doc generator.** `generate-webgpu-operator-md.ts` parses the JSEP registrations (`js_execution_provider.cc`, `js_contrib_kernels.cc`), so post-flip `webgpu-operators.md` describes the wrong EP; point it at the native WebGPU EP registrations (those JSEP sources are removed in Phase 2). diff --git a/docs/design/onnxruntime_web_remove_webgl_backend.md b/docs/design/onnxruntime_web_remove_webgl_backend.md index 6b2d703ada2d2..9a56f2843f7c2 100644 --- a/docs/design/onnxruntime_web_remove_webgl_backend.md +++ b/docs/design/onnxruntime_web_remove_webgl_backend.md @@ -3,7 +3,7 @@ **Scope:** `onnxruntime-web` JavaScript/TypeScript package — WebGL backend **Related work:** -[Migrate onnxruntime-web WebGPU/WebNN from JSEP to the native WebGPU EP](onnxruntime_web_jsep_to_webgpu_ep_migration.md) +[Migrate onnxruntime-web from JSEP to the native WebGPU EP](onnxruntime_web_jsep_to_webgpu_ep_migration.md) — independent, but shares the `onnxruntime-web/all` bundle (§6) and the deprecation-warning utility (§7). --- @@ -110,7 +110,7 @@ a sufficient soft landing. ## 6. `/all` bundle coupling `./all` bundles two independent things: WebGL and the WebGPU/WebNN backend. This effort removes WebGL; the -[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) flips WebGPU/WebNN to the native EP. The +[JSEP → native migration](onnxruntime_web_jsep_to_webgpu_ep_migration.md) flips WebGPU to the native EP. The two are independent and may land in either order; `/all` collapses into a single alias of the webgpu/default bundle once both have landed (whichever lands second performs the repoint). `/all` is kept as an export to avoid breaking imports (see JSEP doc §5). From 3a1ac44c0a0b08e7319a845db05725de3e1a107f Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Fri, 31 Jul 2026 17:49:24 -0700 Subject: [PATCH 10/10] more feedback Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- .../onnxruntime_web_jsep_to_webgpu_ep_migration.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md index ffd669ccc57c0..089b95d4ca9c2 100644 --- a/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md +++ b/docs/design/onnxruntime_web_jsep_to_webgpu_ep_migration.md @@ -115,10 +115,10 @@ migration guide (§11). Parity checks to close before Phase 1. Items 1–3 need a real browser + GPU/WebNN run; item 4 is a known wiring gap to fix. -1. **Proxy-worker (`wasm.proxy = true`).** `./webgpu` already ships this exact path (native EP + Asyncify + proxy +1. **Proxy-worker (`wasm.proxy = true`).** `./webgpu` already ships this exact path (native WebGPU EP + Asyncify + proxy over the EP-agnostic `proxy-wrapper.ts`), so this is a coverage check, not new wiring. CI today runs `--wasm.proxy` only with `-b=wasm` (CPU) on the JSEP build — never `-b=webgpu`, and the `--webgpu-ep` leg runs no proxy. Confirm a - real proxy-mode run on a GPU: native EP device init and threading inside a Worker. + real proxy-mode run on a GPU: native WebGPU EP device init and threading inside a Worker. 2. **IO-binding.** The `'gpu-buffer'` / `'ml-tensor'` paths in `wasm-core-impl.ts` have symmetric native/JSEP hooks. CI today runs the `--io-binding=gpu-tensor` / `gpu-location` legs on the JSEP build only; the `--webgpu-ep` leg passes no `--io-binding`. Confirm the native path delivers true zero-copy handoff and matching dispose/lifetime @@ -137,7 +137,7 @@ Per-session typed options are already a superset of JSEP's, confirmed by source ## 8. Phase 1 — Flip + deprecate (this release) -1. **Flip the default.** Build `.` and `./all` with the native EPs (`USE_WEBGPU_EP=true`). The `webgpu` key and +1. **Flip the default.** Build `.` and `./all` with the native WebGPU EP (`USE_WEBGPU_EP=true`). The `webgpu` key and public API are unchanged. `/all` keeps WebGL until the WebGL effort removes it. 2. **Escape hatch.** Add the temporary, deprecated `onnxruntime-web/jsep` export (`USE_WEBGPU_EP=false`). 3. **Warn once.** The `/jsep` build warns once (respecting `env.logLevel`) via a shared deprecation-warning @@ -160,7 +160,7 @@ does not; and (b) the §7 items resolved with targeted coverage (op-parity tests ## 9. Phase 2 — Removal (subsequent release) Timed one release after Phase 1, keeping the `/jsep` hatch available for exactly one release. Extend only if -native-EP parity regressions surface via real `/jsep` usage. +native-WebGPU-EP parity regressions surface via real `/jsep` usage. 1. Drop JSEP WASM artifacts; update `build.ts` / `package.json`; remove the `USE_WEBGPU_EP` flag. Repoint `/all` to the webgpu/default artifact — only once WebGL has also been dropped (WebGL doc §8); otherwise `/all` stays a @@ -178,9 +178,9 @@ native-EP parity regressions surface via real `/jsep` usage. | Risk | Likelihood | Mitigation | |---|---|---| -| Undiscovered native-EP parity gap vs. JSEP | Medium | One-release `/jsep` escape hatch + warn-once funnel; differential tests | +| Undiscovered native-WebGPU-EP parity gap vs. JSEP | Medium | One-release `/jsep` escape hatch + warn-once funnel; differential tests | | int64 behavior change vs. JSEP | Low | None by default (native-off matches JSEP); `enableInt64 = 1` is an opt-in tradeoff | -| Proxy / IO-binding / WebNN unexercised under the native EP in CI | Unknown | Validate before flip (§7) | +| Proxy / IO-binding / WebNN unexercised in the native-WebGPU build in CI | Unknown | Validate before flip (§7) | | Global `env.webgpu.*` settings dropped on native | Medium | Wire into the native path or document — release gate (§7, §8) | | WASM filename change breaks `wasmPaths` | Medium | Document `.jsep.wasm` → `.asyncify.wasm` | | No telemetry on JSEP adoption | Medium | Warn-once funnel + one-release escape hatch | @@ -195,7 +195,7 @@ native-EP parity regressions surface via real `/jsep` usage. - **Lower-overhead build (`onnxruntime-web/jspi`):** on JSPI-capable browsers, prefer `./jspi` — the same native WebGPU EP with a smaller WASM binary and lower per-call overhead than Asyncify (the universal fallback). - **Need JSEP for one more release:** import `onnxruntime-web/jsep` (temporary, deprecated). File an issue if the - native EPs do not work for your model. + native WebGPU EP does not work for your model. - **int64-heavy models:** no change needed — the default matches JSEP. Exceptions that move int64 arithmetic to the GPU (lossy for genuine `> 2³¹` values): `extra: { 'ep.webgpuexecutionprovider.enableInt64': '1' }`, and `enableGraphCapture = true` (forces int64 on; see §6).