Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: MEMORY LEAK #1120

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

UnlimitedBytes
Copy link

One important caveat was ignored by piping request stream to busboy stream. This commit aknowledges that caveat by closing busboy stream when request stream encounters an error. More information about this caveat can be found on https://nodejs.org/api/stream.html#readablepipedestination-options.

One important caveat is that if the Readable stream emits an error during processing, the Writable destination is not closed automatically. If an error occurs, it will be necessary to manually close each stream in order to prevent memory leaks.

This PR also fixes multiple issues that are caused by this caveat some are listed blow.

Some PR's (that may introduce breaking changes) are closed by this PR also (without breaking changes) and some are listed blow.

PLEASE MERGE THIS BECAUSE IT'S INSANE ANNOYING HAVING TO RELAUNCH THE SERVER EVERY DAY BECAUSE THOUSANDS OF OPEN STREAMS ARE NOT CLOSED (MEMORY LEAKS)

@LinusU here is a not breaking change that will fix all the above.

One important caveat was ignored by piping request stream to busboy stream.
This commit aknowledges that caveat.
More information on: https://nodejs.org/api/stream.html#readablepipedestination-options
As mentioned by @Gerard-Szulc we use `removeListener` in node instead of `removeEventListener`
@import-brain import-brain added the pr label Aug 3, 2022
@@ -42,10 +42,16 @@ function makeMiddleware (setup) {
var pendingWrites = new Counter()
var uploadedFiles = []

function requestError(error) {
if(!busboy) return
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

busboy should always be defined here?

Suggested change
if(!busboy) return

@@ -42,10 +42,16 @@ function makeMiddleware (setup) {
var pendingWrites = new Counter()
var uploadedFiles = []

function requestError(error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
function requestError(error) {
function requestError (error) {

@LinusU
Copy link
Member

LinusU commented Oct 30, 2022

Sorry for the late reply, this looks good 👍

Only have one question, will the next be called with an error when this happens? Could you add a test case covering this?

@titanism
Copy link

Bump

@r4881t
Copy link

r4881t commented Jan 11, 2023

@LinusU Merger?

@titanism
Copy link

@LinusU we can gladly help merge and release here - also see #1161

@iMrDJAi
Copy link

iMrDJAi commented May 15, 2023

@LinusU Can you please merge this? Servers that use multer are exposed to DOS attacks!

@Stanback
Copy link

Bump, can we merge this please?

didiercolens added a commit to didiercolens/bdb-multer that referenced this pull request Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants