Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module: fix error thrown from require(esm) hitting TLA repeatedly #55520

Merged
merged 3 commits into from
Oct 29, 2024

Conversation

joyeecheung
Copy link
Member

@joyeecheung joyeecheung commented Oct 24, 2024

This tracks the asynchronicity in the ModuleWraps when they turn out to contain TLA after instantiation, and throw the right error (ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes the freezing of ModuleWraps since it's not meaningful to freeze this when the rest of the module loader is mutable, and we can record the asynchronicity in the ModuleWrap right after compilation after we get a V8 upgrade that contains v8::Module::HasTopLevelAwait() instead of searching through the module graph repeatedly which can be slow.

Fixes: #55516
Refs: #52697

This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/loaders
  • @nodejs/vm

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. errors Issues and PRs related to JavaScript errors originated in Node.js core. esm Issues and PRs related to the ECMAScript Modules implementation. needs-ci PRs that need a full CI run. labels Oct 24, 2024
Copy link

codecov bot commented Oct 24, 2024

Codecov Report

Attention: Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.

Project coverage is 88.43%. Comparing base (7b5d660) to head (ee2822b).
Report is 26 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/modules/esm/module_job.js 92.85% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #55520      +/-   ##
==========================================
+ Coverage   88.41%   88.43%   +0.01%     
==========================================
  Files         653      654       +1     
  Lines      187476   187690     +214     
  Branches    36083    36126      +43     
==========================================
+ Hits       165763   165987     +224     
- Misses      14946    14949       +3     
+ Partials     6767     6754      -13     
Files with missing lines Coverage Δ
lib/internal/errors.js 96.99% <100.00%> (+<0.01%) ⬆️
lib/internal/modules/esm/loader.js 98.02% <100.00%> (+<0.01%) ⬆️
src/module_wrap.cc 75.87% <100.00%> (-0.11%) ⬇️
lib/internal/modules/esm/module_job.js 99.25% <92.85%> (-0.75%) ⬇️

... and 42 files with indirect coverage changes

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 26, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 26, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 28, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 28, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung joyeecheung added the review wanted PRs that need reviews. label Oct 28, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung joyeecheung added request-ci Add this label to start a Jenkins CI on a PR. and removed review wanted PRs that need reviews. labels Oct 29, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 29, 2024
@nodejs-github-bot
Copy link
Collaborator

@aduh95 aduh95 added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Oct 29, 2024
@joyeecheung joyeecheung added commit-queue Add this label to land a pull request using GitHub Actions. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Oct 29, 2024
@nodejs-github-bot
Copy link
Collaborator

@@ -1650,6 +1650,9 @@ E('ERR_PERFORMANCE_MEASURE_INVALID_OPTIONS', '%s', TypeError);
E('ERR_QUIC_CONNECTION_FAILED', 'QUIC connection failed', Error);
E('ERR_QUIC_ENDPOINT_CLOSED', 'QUIC endpoint closed: %s (%d)', Error);
E('ERR_QUIC_OPEN_STREAM_FAILED', 'Failed to open QUIC stream', Error);
E('ERR_REQUIRE_ASYNC_MODULE', 'require() cannot be used on an ESM ' +
'graph with top-level await. Use import() instead. To see where the' +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think referencing the graph is, whilst completely accurate, too esoteric. I expect many users won't even know what a graph is.

Copy link
Member Author

@joyeecheung joyeecheung Oct 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean the word "graph" in general or using "graph" to mean a module and its dependencies? I believe graph is a very common term used across different programming languages, including referring to the concept of "modules and its dependencies". There are also many module graph visualization tools out there, and its used extensively in materials such as MDN's guide on ESM and V8's user-facing explainer about ESM. If you Google "javascript module graph" you can find numerous tutorials and blog posts introducing ESM mentioning the concept of module graphs, and I suspect many learned about ESM by reading them so they should already know what this is.

@joyeecheung joyeecheung added the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 29, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Oct 29, 2024
@nodejs-github-bot nodejs-github-bot merged commit 8aac7da into nodejs:main Oct 29, 2024
71 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 8aac7da

Ceres6 pushed a commit to Ceres6/node that referenced this pull request Oct 30, 2024
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.

PR-URL: nodejs#55520
Fixes: nodejs#55516
Refs: nodejs#52697
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
RafaelGSS pushed a commit that referenced this pull request Nov 1, 2024
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.

PR-URL: #55520
Fixes: #55516
Refs: #52697
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
louwers pushed a commit to louwers/node that referenced this pull request Nov 2, 2024
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.

PR-URL: nodejs#55520
Fixes: nodejs#55516
Refs: nodejs#52697
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. errors Issues and PRs related to JavaScript errors originated in Node.js core. esm Issues and PRs related to the ECMAScript Modules implementation. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Wrong ERR_REQUIRE_CYCLE_MODULE for require(esm)
7 participants