Skip to content

WIP: Move ESM loaders to worker thread

Rodrigo Muino Tomonari requested to merge github/fork/bmeck/loaders into master
Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

This has some widespread implications:

  • dynamicInstantiate no longer exists since there is no 1st class references between loaders and the thread they are operating on
  • only 1 shared loader is spawned for all the threads it affects, unlike currently where node spins up a new loader on each thread
  • data is done by passing messages which are serialized
  • loaders can no long be affected by mutated globals from non-loader code

This roughly follows some of the older design docs and discussions from @nodejs/modules .

This does not seek to allow having multiple user specified loaders, nor is it seeking to change the loader API signatures, it is purely about moving them off thread and the implications of such.

This does introduce a new type of Worker for loading an internal entry point and also expands the worker_threads API for convenience by allowing a transferList in the workerData to avoid extraneous postMessages.

This will need quite a large writeup on how it works and how data is transferred but this seems a good point to start discussions.

Merge request reports

Loading