-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove multiply-defined environment variables #52
Conversation
I can't remember if there was a specific reason to define the variables in several different places. However, I don't think the definitions have to be made before any DESI module is loaded, and don't anticipate any problems in the environment caused by defining them in the same place as you've done in this branch. As far as I can tell, the comment
applies to the following lines
so you probably want to group those together just below the comment. |
Thank you @marcelo-alvarez, I've actually subdivided those two by the subsection of the I also notice that we do
But 12.2 is the default, so why specifically is this done? |
PS, it would also be nice to have specific documentation for |
I agree it doesn't make sense as is. The @sbailey would know for sure, but it looks like desihub/desiconda#68 removed the desiconda As for With regard to
Given it is now July 2024, and there is no mention of this at the same page, you might assume it's fixed and remove it, though you may want to test it with a test-prod just to make sure. @sbailey should probably make this call. |
Thanks for digging up the history, @marcelo-alvarez . For the purposes of this cleanup update, let's aim for achieving the exact same environment as we currently have, since we know that it works with the current pipeline. i.e. this is about making the modules more maintainable in the future, not about changing the environment now (don't fix what isn't broken). We probably don't need |
I'm OK with leaving the environment variables in place, but I'm still not sure about:
I verified that |
Another question: This version is set as the "default" desiconda version before being overridden on perlmutter. However, this version does not exist at all. Should the default be a version that actually exists?
|
Marking this as ready for review. I've added a few new questions above already. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this cleanup. I verified that at NERSC it produces the same environment as current main, except for
- adding $DESI_TILES (expected)
- PATH has different order due to module unload/load cudatoolkit at the end instead of the beginning (I don't think this matters)
- Miscellaneous module bookkeeping vars like
_ModuleTable011_
and_LMFILES_
(looks ok)
I put several comments inline requesting changes to continue supporting non-NERSC locations like KPNO. I'd also like to confirm with @schlafly and/or @araichoor about how the equivalent of $DESI_TILES is tracked in surveyops / afternoonplanning to know where the fiberassign/tiles/trunk checkout is to make sure we don't re-invent an arbitrarily different wheel.
@sbailey, there's still the question about the default value of |
From Slack #survey-ops discussions, online surveyops has been using $FIBER_ASSIGN_DIR for what we called $DESI_TILES here. I suggest that we switch to using $FIBER_ASSIGN_DIR here (and desihub/desispec#2294) to match that prior usage. @weaverba137 if you feel strongly that we should stick with $DESI_TILES, please document your thoughts here. I could be ok with that, but that doesn't necessarily imply that we would update surveyops to switch to $DESI_TILES instead. i.e. if we go with $DESI_TILES here, it might be better to live with the duplicate names rather than risk breaking operations even briefly for a cosmetic change. |
I don't have any issue with using And oops, I forgot to push my previous commit! |
I've also changed to |
I did some digging on I re-verified that the environment is the same as current main plus $FIBER_ASSIGN_DIR, i.e. looks good. I will merge this now but will do separate testing of this plus desihub/desispec#2294 (and similar other repos) before merging those. |
@sbailey, thank you, I agree with your documentation on |
This PR closes #49.
Important question for @sbailey, @marcelo-alvarez, @akremin, @tskisner: we've decided to consolidate the definition of various NERSC-performance environment variables, e.g.
KMP_INIT_AT_FORK
. Was there a specific reason that the environment variables were defined in several different places in the module file, i.e., the definitions have to be made before at least some DESI modules are loaded?