Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs: fix bufferSize option for opendir recursive #55744

Merged
merged 1 commit into from
Nov 8, 2024

Conversation

Ethan-Arrowood
Copy link
Contributor

@Ethan-Arrowood Ethan-Arrowood commented Nov 6, 2024

The bufferSize option was not respected in recursive mode. This PR
implements a naive solution to fix this issue until a better
implementation can be designed.

Fixes: #48820

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Nov 6, 2024
@Ethan-Arrowood Ethan-Arrowood added the fs Issues and PRs related to the fs subsystem / file system. label Nov 7, 2024
@Ethan-Arrowood Ethan-Arrowood marked this pull request as ready for review November 7, 2024 11:36
@Ethan-Arrowood Ethan-Arrowood added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 7, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 7, 2024
@nodejs-github-bot
Copy link
Collaborator

@Ethan-Arrowood Ethan-Arrowood changed the title fs: fix opendir recursive buffersize issue fs: fix bufferSize option for opendir recursive Nov 7, 2024
@targos targos added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 7, 2024
@Ethan-Arrowood Ethan-Arrowood added commit-queue Add this label to land a pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Nov 7, 2024
@sjohannes
Copy link

Could you make sure to document the issues with this new code (not sure if it's best put in comments or as a ticket)? I assume it's something along the lines of bufferSize now essentially being unlimited within subdirectories? And possibly blocking the event loop while scanning whole directories?

@Ethan-Arrowood
Copy link
Contributor Author

I'll add a comment and open a new issue 👍

@targos targos removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 7, 2024
The bufferSize option was not respected in recursive mode. This PR
implements a naive solution to fix this issue until a better
implementation can be designed.

Fixes: nodejs#48820
@Ethan-Arrowood Ethan-Arrowood added the request-ci Add this label to start a Jenkins CI on a PR. label Nov 7, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Nov 7, 2024
@nodejs-github-bot
Copy link
Collaborator

Copy link

codecov bot commented Nov 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.40%. Comparing base (1aa7135) to head (75b300a).
Report is 16 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #55744   +/-   ##
=======================================
  Coverage   88.40%   88.40%           
=======================================
  Files         654      654           
  Lines      187747   187756    +9     
  Branches    36127    36128    +1     
=======================================
+ Hits       165972   165985   +13     
- Misses      15009    15010    +1     
+ Partials     6766     6761    -5     
Files with missing lines Coverage Δ
lib/internal/fs/dir.js 97.22% <100.00%> (+0.03%) ⬆️

... and 28 files with indirect coverage changes

@Ethan-Arrowood Ethan-Arrowood added the commit-queue Add this label to land a pull request using GitHub Actions. label Nov 8, 2024
@Ethan-Arrowood
Copy link
Contributor Author

CI is passing except for some flaky tests as far as I can tell.

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot nodejs-github-bot added commit-queue-failed An error occurred while landing this pull request using GitHub Actions. and removed commit-queue Add this label to land a pull request using GitHub Actions. labels Nov 8, 2024
@nodejs-github-bot
Copy link
Collaborator

Commit Queue failed
- Loading data for nodejs/node/pull/55744
✔  Done loading data for nodejs/node/pull/55744
----------------------------------- PR info ------------------------------------
Title      fs: fix bufferSize option for opendir recursive (#55744)
Author     Ethan Arrowood <[email protected]> (@Ethan-Arrowood)
Branch     Ethan-Arrowood:fix/issue-48820 -> nodejs:main
Labels     fs, test, needs-ci
Commits    1
 - fs: fix bufferSize option for opendir recursive
Committers 1
 - Ethan Arrowood <[email protected]>
PR-URL: https://github.com/nodejs/node/pull/55744
Fixes: https://github.com/nodejs/node/issues/48820
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/55744
Fixes: https://github.com/nodejs/node/issues/48820
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
--------------------------------------------------------------------------------
   ⚠  Commits were pushed since the last approving review:
   ⚠  - fs: fix bufferSize option for opendir recursive
   ℹ  This PR was created on Wed, 06 Nov 2024 12:12:12 GMT
   ✔  Approvals: 2
   ✔  - Yagiz Nizipli (@anonrig) (TSC): https://github.com/nodejs/node/pull/55744#pullrequestreview-2420753399
   ✔  - Michaël Zasso (@targos) (TSC): https://github.com/nodejs/node/pull/55744#pullrequestreview-2420811043
   ✔  Last GitHub CI successful
   ℹ  Last Full PR CI on 2024-11-08T12:03:25Z: https://ci.nodejs.org/job/node-test-pull-request/63487/
- Querying data for job/node-test-pull-request/63487/
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu
https://github.com/nodejs/node/actions/runs/11742390923

@Ethan-Arrowood Ethan-Arrowood added commit-queue Add this label to land a pull request using GitHub Actions. and removed commit-queue-failed An error occurred while landing this pull request using GitHub Actions. labels Nov 8, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Nov 8, 2024
@nodejs-github-bot nodejs-github-bot merged commit e0ef622 into nodejs:main Nov 8, 2024
67 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in e0ef622

@Ethan-Arrowood Ethan-Arrowood deleted the fix/issue-48820 branch November 11, 2024 17:41
@Ethan-Arrowood
Copy link
Contributor Author

Ethan-Arrowood commented Nov 11, 2024

Should we consider backporting this since it is a bug fix affecting all current versions?

aduh95 pushed a commit that referenced this pull request Nov 16, 2024
The bufferSize option was not respected in recursive mode. This PR
implements a naive solution to fix this issue until a better
implementation can be designed.

Fixes: #48820
PR-URL: #55744
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fs Issues and PRs related to the fs subsystem / file system. needs-ci PRs that need a full CI run. test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

fs#opendir does not return all files in subdirectories when bufferSize is less than the directory size
5 participants