Script-based configuration files in "type": "module" packages

[guybedford]

Tracking issue specifically for the ESLint issue that was raised here - eslint/eslint#12319.

Tooling often rely on configuration scripts. Examples include .eslintrc.js, karma.conf.js, webpack.config.js, babel.config.js to name a few.

These tools typically load those configuration files with a require.

When an existing project with such a configuration adds "type": "module" to the package.json, they get the ERR_REQUIRE_ESM dual hazard error that we added recently in nodejs/node#29492.

The ideal upgrade path is to have the tools support using import() for loading their configurations. Alternatively they could support .cjs as well.

We should probably reach out with this information to tools and get further feedback as well.

[coreyfarrell]

This is a major breaking change for nyc and tap. Both supported testing ES modules using the esm module or with @babel/register. Now running nyc and/or tap on node.js 12.11.0 with "type": "module" causes a complete failure of testing, both fail with Error [ERR_REQUIRE_ESM]: Must use import to load ES Module.

CC @bcoe @isaacs

[Jamesernator]

Perhaps once again consider a pragma-based approach as an escape hatch (e.g. "use commonjs" or "format:commonjs").

While there's a little burden on package users it gives them a functioning solution until the tools can update.

[guybedford]

I've added nodejs/node#29732 to put this check back behind the --experimental-modules flag, instead of being on by default, while we discuss it further.

[GeoffreyBooth]

eslint/eslint#12319 (comment)

[devsnek]

i'd like to really strongly advocate that we look for solutions that reduce the occurrences of package.json files. they slow loading down quite a bit (which is why we traditionally never supported any fields besides "main") and are obviously incompatible with non-node runtimes.

[coreyfarrell]

The ideal upgrade path is to have the tools support using import() for loading their configurations.

https://nodejs.org/api/esm.html#esm_code_import_code_expressions suggests that dynamic import() can be used from CommonJS but this is not always true. In node.js 8 if --experimental-modules is not enabled then import(specifier) is a SyntaxError. It's always a SyntaxError in node.js 6 (I know node.js does not support 6.x but some tools still do). In node.js 10/12 it rejects asynchronously if --experimental-modules is not enabled. Could the dynamic import function be conditionally exposed so it could be grabbed from say require.dynamicImport? Obviously dynamicImport should only be defined if --experimental-modules is enabled or unflagged. This would allow tools to detect the feature without risk of SyntaxError or having to deal with asynchronous 'Not supported' rejections.

This does not resolve the issue with test runners executing scripts, I'm not sure how that can be solved.

[devsnek]

it's a syntax error the same way trying to use numeric separators in node 8 is a syntax error. i don't think we should try to circumnavigate that.

[coreyfarrell]

Numeric separators throwing SyntaxError in node 8 is not an issue because not using numeric separators is never a SyntaxError. A way to load *.config.js is needed, I'm just pointing out that the suggestion to use import() is not practical as it is a syntax error by default in all current versions of node.js.

[devsnek]

import is syntax because it's not just a function call, it has side-channel data about where it's called from that is needed to resolve import requests.

aside from that though, one of our goals is to make node code more interoperable by default, and adding some new import api would interfere with that.

[ljharb]

I think the point @coreyfarrell is trying to make (that i agree with) is that since most ecosystem packages support "not just the latest node", it is simply not a viable workaround to recommend use of import() - additionally, since older nodes won't support requiring of .cjs, that is also not a sufficient workaround.

[bmeck]

@ljharb require loads any unknown extension as CJS, .cjs works for all versions of node as long as someone doesn't clobber require.extensions['.cjs'].

[ljharb]

Thanks for the correction; in that case, that's probably the best workaround for the time being.

[coreyfarrell]

import is syntax because it's not just a function call, it has side-channel data about where it's called from that is needed to resolve import requests.

aside from that though, one of our goals is to make node code more interoperable by default, and adding some new import api would interfere with that.

It's true that tools can add support for loading config from .cjs files. This does not help move forward. Let's say that --experimental-modules gets unflagged in node.js 12.12.0 (first LTS). It will be years before these tools drop support for everything older than v12 LTS. I'd like to see these tools support ESM configuration files but without a non-syntax method of importing ESM this will not happen.

Would it reduce your concerns if module.createRequire never provided the dynamicImport method? That way require.dynamicImport(filename) would only be available from CJS, never ESM.

[nicolo-ribaudo]

@coreyfarrell I'm thinking about this for Babel:

async function tryLoad(path) {
  try {
    return require(path);
  } catch (e) {
    try {
      return await require("./import")(path);
    } catch {
       throw e;
    }
  }
}


// import.js
module.exports = p => import(p);

By doing so, the file containing import isn't loaded in old node.