Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce memory usage in io proxy process #9424

Merged
merged 1 commit into from
Feb 14, 2025

Conversation

dgud
Copy link
Contributor

@dgud dgud commented Feb 12, 2025

Heap grows a lot with large printouts and stays large, which causes problems when running in testcases in parallel, which keeps a lot of these processes around.

In my case the used memory went from 6.6GB to 500MB.

Heap grows a lot with large printouts and stays large, which causes
problems when running in testcases in parallel, which keeps a lot of
these processes around.

In my case the used memory went from 6.6GB to 500MB.
@dgud dgud added the team:PS Assigned to OTP team PS label Feb 12, 2025
@dgud dgud requested review from u3s and Whaileee February 12, 2025 13:16
@dgud dgud self-assigned this Feb 12, 2025
Copy link
Contributor

github-actions bot commented Feb 12, 2025

CT Test Results

  2 files   57 suites   1h 16m 24s ⏱️
451 tests 438 ✅ 13 💤 0 ❌
485 runs  469 ✅ 16 💤 0 ❌

Results for commit b6e6804.

♻️ This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

// Erlang/OTP Github Action Bot

@dgud dgud added the testing currently being tested, tag is used by OTP internal CI label Feb 12, 2025
@michalmuskala
Copy link
Contributor

Have you tried changing the fullsweep_after setting of a process - this should force more frequent major GC too, without sprinkling the code with manual calls to GC

@dgud
Copy link
Contributor Author

dgud commented Feb 13, 2025

No didn't think of that, I had 20-30 of these 200MB processes that caused the 32b runs to crash with out of memory so
might have gone for the largest axe around, but these processes have almost no live data, so the fullsweep GC is cheap and in my case the overall time was slightly faster.
Changing to minor GC didn't help at all in this case.

But will try you suggestion before merging.

@dgud
Copy link
Contributor Author

dgud commented Feb 14, 2025

Yeah that didn't help that much, only down to ~3GB.

@dgud dgud merged commit d4c6e78 into erlang:maint Feb 14, 2025
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team:PS Assigned to OTP team PS testing currently being tested, tag is used by OTP internal CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants