Skip to content

module: __cjsModule on ESM CJS wrapper and in require ESM

Rodrigo Muino Tomonari requested to merge github/fork/guybedford/esm-cjs-marker into main

As a follow-up to https://github.com/nodejs/node/pull/52166 which explicitly adds support for the __esModule wrapper marker for ESM modules imported into CJS, this implements a complementary __cjsModule wrapper marker for the ESM wrappers created when importing CJS modules into ESM. In addition, when require(esm) is applied to any module exporting __cjsModule: true (including these wrappers), it will handle unwrapping the default supporting custom CJS values and transitive reexports of wrappers exposing their underlying CommonJS modules.

This came out of lengthy discussion with @joyeecheung both online and offline around that PR working through various ecosystem implications, and seems to me to be a good compromise to any potential interop problems https://github.com/nodejs/node/pull/52166 may cause in future.

The use case here is that given an arbitrary namespace object imported into a JavaScript file, in dynamic linking workflows where no further information is available, it is useful to be able to infer many interop patterns against such arbitrary modules.

For example, consider an ESM module that is transpiled into a CommonJS module and then transpiled back to ESM running on the web, where a dependency could be in an unknown module format:

Faux ESM:

import dep from 'dep';
console.log(dep);

Transpiled from ESM to CJS (as tools do today, and then published to npm say):

const depMod = require('dep');
const depNs = depMod.__esModule ? depMod : { default: depMod };
console.log(depNs.default);

Now if I run this module in Node.js, dep may now get an __esModule marker added by Node.js if it doesn't have an __esModule marker already and has a default export.

So if I want to transpile this module back into ESM and run it on the web, with that same dynamic dependency linkage, I end up with something like:

import * as depMod from 'dep';

// New interop layer to mimic Node.js require ESM semantics (in reality you'd use a proxy for live bindings)
const depRequire = !('__esModule' in depMod) && ('default' in depMod) ? { ...depMod, __esModule: true } : depMod;

// Original published CJS, with require value replaced:
const depMod = depRequire;
const depNs = depMod.__esModule ? depMod : { default: depMod };
console.log(depNs.default);

So we now have a scheme to transpile arbitrary CJS to ESM for browsers against full dynamic externals by adding this new interop layer where an arbitrary require('dep') in a build to ESM gets turned into:

import * as depMod from 'dep';
const depRequire = !('__esModule' in depMod) && ('default' in depMod) ? { ...depMod, __esModule: true } : depMod;

That is, even if we don't know the value of dep, we have a way we can coerce it into the correct value and interop with it correctly, regardless of whether it came from CommonJS, ESM or faux ESM.

But there is now a new interop problem with such a scheme - in the case where depMod is a CJS module transpiled into ESM, we may want the CJS transpiled into ESM to be represented by the Node.js CJS ESM wrapper which looks like { default: module.exports, ...namedExportsFromCjsModuleLexer }.

But if we pass a module of the above shape in the above interop pattern as depMod, since it does not have an __esModule and it does have a default, it will give a value of depRequire being { default: module.exports, ...namedExportsFromCjsModuleLexer, __esModule: true }, when we would have expected depRequire to have been module.exports!

But, if there is a __cjsModule marker in the CJS ESM representation, then the interop can be updated to be:

import * as depMod from 'dep';
const depRequire = depMod.__cjsModule ? depMod.default : !('__esModule' in depMod) && ('default' in depMod) ? { ...depMod, __esModule: true } : depMod;

Therefore fully solving the dynamic externalization against all of the cases where dep might respectively be CJS transpiled to ESM and real ESM with the conditional faux ESM interop layer for the consuming faux ESM as CJS transpiled into ESM per https://github.com/nodejs/node/pull/52166.

I understand at this point that this all may seem contrived, but in real-world interop patterns, there is no concept of reaching some compositional complexity limit - users will apply tools to code, and expect it to work. Fractal interop is the name of the game. And where the past interop concerns were about ensuring ESM semantics transpiled to CJS, our future interop concerns are now the semantics of CJS transpiled to ESM. __cjsModule helps to bridge that much like __esModule did when I originally implemented it in Traceur and Babel later adopted.

By landing and porting this alongside https://github.com/nodejs/node/pull/52166, it should give another tool to interop authors wherever they are in the toolchain to be able to have a way to deal with these kinds of onion interops. And while the exact interop issue is not something that Node.js itself has to deal with, I strongly believe that being friendly to tooling dealing with hard interop cases would be a net win for the ecosystem. Well that, and also I very much need this feature in my own tooling workflows.

Note that this would be a major change, and should go out at the same time as the unflagging for require(esm).

//cc @nodejs/loaders

Merge request reports