-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make heap size hint available as an env variable #55631
base: master
Are you sure you want to change the base?
Conversation
Bump; this would be nice |
It would be nice if the envvar could also be added to the docs 🙂 |
Where should it go? It wasn't clear to me where to put it. |
@tecosaur I couldn’t find documentation on |
On https://docs.julialang.org/en/v1/manual/environment-variables/, all ENV variables are explained. So I would look for it there to find more information |
Done. |
Ping. Would be nice to have this. |
- b (bytes) | ||
- k (kibibytes) | ||
- m (mebibytes) | ||
- g (gibibytes) | ||
- t (tebibytes) | ||
- % (percentage of physical memory) | ||
|
||
which are not case sensitive. For example, `JULIA_HEAP_SIZE_HINT=1G` would provide a 1 GB heap size hint to the garbage collector. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The units seem a bit inconsistent.. lowercase in the top list, uppercase in the example, then two-letter uppercase in the cli docs
Lines 234 to 237 in e1e7763
" --heap-size-hint=<size> Forces garbage collection if memory usage is higher\n" | |
" than the given value. The value may be specified as a\n" | |
" number of bytes, optionally in units of KB, MB, GB,\n" | |
" or TB, or as a percentage of physical memory with %.\n\n" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just for reference – the current method in Julia, for whatever reason, is written as to not be case sensitive and actually only takes the first character – so, e.g., 5GB
would get parsed as 5g
. So technically these are all valid. For example:
julia --heap-size-hint=1giga
will successfully set a 1 GB heap size hint. This is totally valid (though not documented, I guess?) on the current Julia.
I personally don't have a preference other than I agree consistency is always good. Wdyt? In retrospect I think the parsing should have required a specific case and number of characters. But too late for that now as all Julia from 1.9+ have this quirk.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The inconsistency is also found in the existing docs. e.g., the current code is
" number of bytes, optionally in units of KB, MB, GB,\n"
" or TB, or as a percentage of physical memory with %.\n\n"
while other docs give the single-letter version: https://julialang.org/blog/2023/04/julia-1.9-highlights/#memory_usage_hint_for_the_gc_with_--heap-size-hint
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would've thought matching here would make most sense https://docs.julialang.org/en/v1.12-dev/manual/command-line-interface/ so uppercase 2 chars. (Though I see 1.10 does give 'G` as an example.. but better to match current version)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok will do the 2-char version, upper-case, everywhere.
Except B and % as those are single chars
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, I take that back. It looks weird. Note that most unix tools, a single upper-case character is used. For example, dd
, rsync
, ulimit
- they all use K for kilobytes, G for gigabytes, M for megabytes.
All slurm commands use this convention too.
Even ls
will display file sizes with a single character unit: ls -l
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok I made everything single char in the docs. (Note that it's backwards compatible; it's just the docs were inconsistent and ambiguous before)
Tagging triage to get eyes on this and because it adds new API in the form of an |
This makes
JULIA_HEAP_SIZE_HINT
the environment variable version of the--heap-size-hint
command-line flag. Seems like there was interest in #45369 (comment).The same syntax is used as for the command-line version with, for example,
2G
=> 2 GB and200M
=> 200 MB.@oscardssmith want to take a look?