Skip to content

Commit 51800ac

Browse files
privatenumberprivatenumber
authored
docs: add Node internals knowledge base (notes/node)
1 parent a305f36 commit 51800ac

7 files changed

Lines changed: 424 additions & 0 deletions

File tree

‎notes/node/README.md‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# Node.js internals notes
2+
3+
Maintainer notes tracking how Node module-loading internals affect tsx. These are not user docs (`files: ["dist"]` keeps them out of the npm package).
4+
5+
tsx fills gaps in Node's TypeScript and module-loading story. As Node adds native support — type stripping, `require(esm)`, sync module hooks — those gaps get smaller, which is the preferred direction. These notes keep tsx aligned with Node behavior and identify the remaining gaps where tsx still helps users today.
6+
7+
## Reading order
8+
9+
### Mechanism stories
10+
11+
| File | Covers |
12+
| --- | --- |
13+
| [module-hooks.md](./module-hooks.md) | Async `module.register()` vs sync `module.registerHooks()`, loader-worker cost, CJS-reload safety |
14+
| [cjs-loader.md](./cjs-loader.md) | CJS resolution, cache identity, `require.extensions`, eager ESM error decoration |
15+
| [cjs-esm-interop.md](./cjs-esm-interop.md) | ESM importing CJS, CJS requiring ESM, named-export preparsing, `module.exports` interop |
16+
| [type-stripping.md](./type-stripping.md) | Node's native TypeScript pipeline, limitations, current issues, tsconfig contract |
17+
18+
### Reference maps
19+
20+
| File | Covers |
21+
| --- | --- |
22+
| [gate-reference.md](./gate-reference.md) | Smaller gates: import attributes, `import.meta` props, wasm, package main, test runner flags |
23+
| [node-integration-points.md](./node-integration-points.md) | Node loader integration points to re-verify across releases |
24+
25+
## Gate index
26+
27+
| `src/utils/node-features.ts` gate | Context |
28+
| --- | --- |
29+
| `moduleRegister` | [module-hooks.md](./module-hooks.md#moduleregister--async-hooks-loader-worker-thread) |
30+
| `moduleRegisterHooksCjsReload` | [module-hooks.md](./module-hooks.md#cjs-reload-safety--the-boundary-tsx-actually-gates-on) |
31+
| `importAttributes` | [gate-reference.md](./gate-reference.md#import-attributes-importattributes) |
32+
| `testRunnerGlob` | [gate-reference.md](./gate-reference.md#test-runner-glob-testrunnerglob) |
33+
| `cliTestFlag` | [gate-reference.md](./gate-reference.md#--test-flag-clitestflag) |
34+
| `esmLoadReadFile` | [cjs-esm-interop.md](./cjs-esm-interop.md#esm-importing-cjs) |
35+
| `importMetaPathProperties` | [gate-reference.md](./gate-reference.md#importmeta-path-properties-importmetapathproperties) |
36+
| `requireEsm` | [cjs-esm-interop.md](./cjs-esm-interop.md#cjs-requiring-esm) |
37+
| `requireEsmNoWarning` | [cjs-esm-interop.md](./cjs-esm-interop.md#cjs-requiring-esm) |
38+
| `cjsNamespaceModuleExports` | [cjs-esm-interop.md](./cjs-esm-interop.md#cjs-requiring-esm) |
39+
| `nativeTypeScript` | [type-stripping.md](./type-stripping.md#timeline) |
40+
| `wasmModules` | [gate-reference.md](./gate-reference.md#wasm-modules-wasmmodules) |
41+
| `cjsNamespaceFromLoadHook` | [cjs-esm-interop.md](./cjs-esm-interop.md#esm-importing-cjs) |
42+
| `requireEsmExtensionlessMjs` | [cjs-esm-interop.md](./cjs-esm-interop.md#cjs-requiring-esm) |
43+
| `modulePackageMainResolution` | [gate-reference.md](./gate-reference.md#legacymainresolve-assertion-fix-modulepackagemainresolution) |
44+
45+
## Entry format
46+
47+
Each documented behavior should record:
48+
49+
1. what changed in Node;
50+
2. the Node PR or issue;
51+
3. verified versions — first release per major line, confirmed from Node git history;
52+
4. exact tagged Node source anchors (`github.com/nodejs/node/blob/<tag>/...#L...`), ideally last-without and first-with for boundaries;
53+
5. tsx code paths affected;
54+
6. coverage when relevant.
55+
56+
Optional sections:
57+
58+
- `## Decisions` for untestable tradeoffs and rejected alternatives;
59+
- `## Implementation history in tsx` for old commits/PRs that explain why the current approach exists.
60+
61+
Testable behavior belongs in `tests/`; these notes explain context and maintenance intent.
62+
63+
## Verification workflow
64+
65+
Version boundaries are pinned from Node's git history, because backports often land after the main-line commit.
66+
67+
```text
68+
cd /path/to/nodejs/node
69+
git fetch --tags
70+
71+
# Every commit carrying a PR-URL trailer = main-line commit + all backport
72+
# cherry-picks. Collect all release tags containing them, first per major line:
73+
pr=59929
74+
{ for sha in $(git log --grep "PR-URL: https://github.com/nodejs/node/pull/${pr}\$" --format=%H --all); do
75+
git tag --contains "$sha" | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$'
76+
done; } | sort -u -V | awk -F. '{maj=$1} maj!=prev{print; prev=maj}'
77+
```
78+
79+
After finding the boundary, verify the exact source shape with `git show <tag>:<path>` and link the public GitHub URL. Do not link to moving branches for source claims.
80+
81+
Verification caveats:
82+
83+
- A PR can be the **fix** to a feature, not its **introduction**. When PR-contains and source-reading disagree, read the source at the tag and document both.
84+
- A fix backported only to an older line means newer lines may never have had the bug; model that as a range or a line-specific gate.
85+
- Some gates describe a bug window `[from, before)` — opened by one PR, closed by another.

‎notes/node/cjs-esm-interop.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# CJS/ESM interop
2+
3+
How Node bridges CommonJS and ES modules, and how tsx shapes transformed TypeScript so it still fits Node's CJS/ESM contracts.
4+
5+
## Shared foundation: CJS preparse + synthetic namespaces
6+
7+
When ESM imports CJS, Node synthesizes an ESM namespace from static CJS source analysis. It does **not** discover named exports by running the CJS module first.
8+
9+
- In v24.15.0, the CJS translator calls `cjsPreparseModuleExports` before building the wrapper ([`translators.js#L212`](https://github.com/nodejs/node/blob/v24.15.0/lib/internal/modules/esm/translators.js#L212)); the function is defined at [`translators.js#L381`](https://github.com/nodejs/node/blob/v24.15.0/lib/internal/modules/esm/translators.js#L381).
10+
- The parser returns `[exports, reexports]` ([`translators.js#L393`](https://github.com/nodejs/node/blob/v24.15.0/lib/internal/modules/esm/translators.js#L393)). Node follows reexports such as `module.exports = { ...require('x') }` recursively ([`translators.js#L398-L418`](https://github.com/nodejs/node/blob/v24.15.0/lib/internal/modules/esm/translators.js#L398-L418)).
11+
- Node marks CJS modules cached by the ESM loader with `kIsCachedByESMLoader` ([`translators.js#L368`](https://github.com/nodejs/node/blob/v24.15.0/lib/internal/modules/esm/translators.js#L368)); the CJS loader uses that marker during circular loads ([`loader.js#L1297-L1308`](https://github.com/nodejs/node/blob/v24.15.0/lib/internal/modules/cjs/loader.js#L1297-L1308)).
12+
- Node v23+ adds a synthetic `'module.exports'` key to the namespace wrapper ([v23.0.0 `translators.js#L187`](https://github.com/nodejs/node/blob/v23.0.0/lib/internal/modules/esm/translators.js#L187)); newer versions preserve it unless the lexer already found that name ([v24.15.0 `translators.js#L219-L220`](https://github.com/nodejs/node/blob/v24.15.0/lib/internal/modules/esm/translators.js#L219-L220)).
13+
14+
The parser implementation changed but the grammar contract stayed the same:
15+
16+
- v24.11.1 uses vendored JS `cjs-module-lexer` ([`translators.js#L73-L82`](https://github.com/nodejs/node/blob/v24.11.1/lib/internal/modules/esm/translators.js#L73-L82)). Its README documents supported patterns (`exports.foo`, `module.exports = { a }`, reexports) at [`deps/cjs-module-lexer/README.md#L7-L51`](https://github.com/nodejs/node/blob/v24.11.1/deps/cjs-module-lexer/README.md#L7-L51) and scope-insensitive matching at [`README.md#L147-L152`](https://github.com/nodejs/node/blob/v24.11.1/deps/cjs-module-lexer/README.md#L147-L152).
17+
- v24.14.0 switches to native `internalBinding('cjs_lexer')` via [nodejs/node#61456](https://github.com/nodejs/node/pull/61456) ([`translators.js#L72`](https://github.com/nodejs/node/blob/v24.14.0/lib/internal/modules/esm/translators.js#L72)). Merve documents the same export/reexport grammar at [`merve.h#L82-L102`](https://github.com/nodejs/node/blob/v24.14.0/deps/merve/merve.h#L82-L102), parses exports/reexports in [`merve.cpp#L630-L676`](https://github.com/nodejs/node/blob/v24.14.0/deps/merve/merve.cpp#L630-L676), and returns the `[Set, Array]` shape from [`src/node_cjs_lexer.cc#L44-L87`](https://github.com/nodejs/node/blob/v24.14.0/src/node_cjs_lexer.cc#L44-L87).
18+
19+
tsx depends on the grammar, not the implementation. Grammar changes are what to re-check when verifying named-export interop on a new Node line.
20+
21+
## ESM importing CJS
22+
23+
Two gates control this direction:
24+
25+
| Gate | Node change | Node PR(s) | Verified releases | tsx use |
26+
| --- | --- | --- | --- | --- |
27+
| `esmLoadReadFile` | The ESM `load` hook can return source for `format === 'commonjs'`. | [nodejs/node#50825](https://github.com/nodejs/node/pull/50825) | v20.11.0, v21.3.0 | `src/esm/hook/load.ts` can transform CJS before Node evaluates/preparses it. |
28+
| `cjsNamespaceFromLoadHook` | CJS source returned by the load hook can be preparsed into a namespace. | [nodejs/node#50825](https://github.com/nodejs/node/pull/50825), [#54769](https://github.com/nodejs/node/pull/54769) | `[20.11.0, 21.0.0)` and `>=21.3.0` | tsx can preserve named exports from transformed CommonJS TypeScript. |
29+
30+
Node's load hook passes import attributes context and reads CJS source at the boundary ([v20.11.0 `load.js#L113-L124`](https://github.com/nodejs/node/blob/v20.11.0/lib/internal/modules/esm/load.js#L113-L124), [`load.js#L145`](https://github.com/nodejs/node/blob/v20.11.0/lib/internal/modules/esm/load.js#L145); [v21.3.0 `load.js#L145`](https://github.com/nodejs/node/blob/v21.3.0/lib/internal/modules/esm/load.js#L145)). v20.11.0's translator then preparses CJS before namespace creation ([`translators.js#L190`](https://github.com/nodejs/node/blob/v20.11.0/lib/internal/modules/esm/translators.js#L190)) and evaluates via `CJSModule._load` ([`translators.js#L203`](https://github.com/nodejs/node/blob/v20.11.0/lib/internal/modules/esm/translators.js#L203)).
31+
32+
tsx uses this by transforming TypeScript to JavaScript before Node preparses it. The important emitted shape is esbuild's dead-code CJS export annotation:
33+
34+
```text
35+
0 && (module.exports = { namedExport });
36+
```
37+
38+
The annotation never runs, but Node's CJS lexer recognizes it. `parentImportsCommonJsExports` detects when a parent imports named or namespace exports from a CJS target; `resolve.ts` adds `tsx-commonjs-export-preparse=1`; `load.ts` sees that query and returns transformed JavaScript so Node's preparse step can build the right namespace. The query is stripped from user-visible URLs after it has coordinated resolve/load/cache behavior.
39+
40+
## CJS requiring ESM
41+
42+
Node's `require(esm)` support is a set of overlapping windows:
43+
44+
| Gate | Node behavior | Node PR(s) | Verified releases / window | tsx use |
45+
| --- | --- | --- | --- | --- |
46+
| `requireEsm` | CJS `require()` can load eligible ESM instead of throwing `ERR_REQUIRE_ESM`. Before: `.mjs` throws ([v20.18.0 `loader.js#L1285`](https://github.com/nodejs/node/blob/v20.18.0/lib/internal/modules/cjs/loader.js#L1285)); after: `loadESMFromCJS` handles it ([v20.19.0 `loader.js#L1310`](https://github.com/nodejs/node/blob/v20.19.0/lib/internal/modules/cjs/loader.js#L1310), [`loader.js#L1509-L1511`](https://github.com/nodejs/node/blob/v20.19.0/lib/internal/modules/cjs/loader.js#L1509-L1511)). | [nodejs/node#55085](https://github.com/nodejs/node/pull/55085) | v20.19.0, v22.12.0, v23.0.0 | Enables native-style `require(esm)` interop in transformed TS. |
47+
| `requireEsmNoWarning` | Normal `require(esm)` stops printing the experimental warning. v22.12 emits it ([`loader.js#L1404`](https://github.com/nodejs/node/blob/v22.12.0/lib/internal/modules/cjs/loader.js#L1404)); v22.13 keeps it behind tracing ([`loader.js#L1401-L1405`](https://github.com/nodejs/node/blob/v22.13.0/lib/internal/modules/cjs/loader.js#L1401-L1405)). | [nodejs/node#56194](https://github.com/nodejs/node/pull/56194) | v20.19.0, v22.13.0, v23.5.0 | Version-sensitive warning expectations. |
48+
| `requireEsmExtensionlessMjs` | Bug window for extensionless specifiers resolving to `.mjs`. Broken window filters `.mjs` from extensionless lookup ([v20.19.0 `loader.js#L440`](https://github.com/nodejs/node/blob/v20.19.0/lib/internal/modules/cjs/loader.js#L440), [`loader.js#L654-L658`](https://github.com/nodejs/node/blob/v20.19.0/lib/internal/modules/cjs/loader.js#L654-L658); [v22.12.0 `loader.js#L676-L680`](https://github.com/nodejs/node/blob/v22.12.0/lib/internal/modules/cjs/loader.js#L676-L680)). Fixed by passing resolved format/source to `loadESMFromCJS` ([v20.19.5 `loader.js#L1303`](https://github.com/nodejs/node/blob/v20.19.5/lib/internal/modules/cjs/loader.js#L1303), [`loader.js#L1503`](https://github.com/nodejs/node/blob/v20.19.5/lib/internal/modules/cjs/loader.js#L1503); [v22.14.0 `loader.js#L1325`](https://github.com/nodejs/node/blob/v22.14.0/lib/internal/modules/cjs/loader.js#L1325), [`loader.js#L1536`](https://github.com/nodejs/node/blob/v22.14.0/lib/internal/modules/cjs/loader.js#L1536)). | [nodejs/node#55085](https://github.com/nodejs/node/pull/55085), [#55590](https://github.com/nodejs/node/pull/55590) | `[20.19.0, 20.19.5)`, `[22.12.0, 22.14.0)` | `isFeatureSupportedInRange` gate for the broken window only. |
49+
| `cjsNamespaceModuleExports` | Synthetic CJS namespaces expose `'module.exports'`. | [nodejs/node#57366](https://github.com/nodejs/node/pull/57366) later fixes this area | v23.0.0 feature | Supports tsx's `module.exports` unwrap semantics. |
50+
51+
For transformed ESM required from CJS, tsx mirrors Node's `export { value as "module.exports" }` escape hatch. esbuild emits ESM exports as accessor descriptors; ordinary CJS object-literal exports are data descriptors. `src/cjs/api/module-extensions.ts` reads `Object.getOwnPropertyDescriptor(exports, 'module.exports')` and unwraps only when the descriptor is a getter and the file is a native `require(esm)` candidate. This keeps ordinary CJS objects with a literal `"module.exports"` key intact.
52+
53+
## Decisions
54+
55+
tsx lets Node own namespace construction wherever possible. It transforms TypeScript only far enough for Node's own lexer/translator to see JavaScript that matches Node's documented internal grammar. That is why the implementation prefers transform-before-preparse and descriptor-based unwrapping over a parallel namespace construction algorithm.
56+
57+
## Implementation history in tsx
58+
59+
- [7c85303](https://github.com/privatenumber/tsx/commit/7c85303) introduced named import from CJS support: the key issue was Node namespace construction, not CJS execution.
60+
- [807f467](https://github.com/privatenumber/tsx/commit/807f467) showed why decorator/TypeScript syntax must be transformed before the lexer sees it.
61+
- [11de737](https://github.com/privatenumber/tsx/commit/11de737) kept export annotation on the original file URL and returned transformed source so Node can preparse named exports while preserving relative resolution and `import.meta.url`; it also added the parent-import-shape detection that became `parentImportsCommonJsExports`.
62+
- [cf8f199](https://github.com/privatenumber/tsx/commit/cf8f199) added descriptor-based `module.exports` unwrapping for native `require(esm)` parity while avoiding false positives for ordinary CJS object keys.

0 commit comments

Comments
 (0)