Official GACODE build for OMFIT users

[jcandy]

@wdeshazer I am making this issue as a place to discuss the model for deploying GACODE in a way that ensures code pushed to GACODE is immediately available in OMFIT.

[smithsp]

You would like the version of OMFIT to be synced to the version of gacode?

[jcandy]

I guess the best case scenario would be that OMFIT users can see updates to GACODE immediately after a pull (for python code), or immediately after pull/make (for fortran code). For non-OMFIT (GACODE only) users, I'd like to see them use the official build system.

[wdeshazer]

So @jcandy my understanding of your concern now is two fold -

1. pygacode adds a layer on top of GACODE that you would prefer not be there.
2. OMFIT's utilization of GACODE is
   • Too static
   • Doesn't allow for transmission of real time updates and branch development

Along those lines, I throw in my own 2 cents which I know could totally be your point
- Working with [:edit] custom GACODE repos [:edit] is manual and requires more domain knowledge than it should have to
- You have to download the repo from github
- You then have to tell OMFIT where to look for it
- If you want to share it with anyone, you have to do that outside of omfit
- The have to repeat the steps you did to make it available to them

There are probably more problems, but those are pretty good.

I belive the solution is to make GACODE a submodule of OMFIT. _In many many repos, t_his is the current solution. The idea is: If my code has a separate mission from your code, but my code is dependent on your code and needs to be aware of the latest and greatest changes to have meaningful co-development to be functional then incorporate it as a submodule. That's what a submodule does. (Matplotlib, numpy, and many non Python based, but I can't think of)

As an aside, that's what I think we should do with OpenBLAS, since GACODE will always be dependent on a custom build. Then the source will be deployed with the build.

But the submodules allows:

• Transmission of the GACODE repo with every deployment of OMFIT
• Hash matching between version in a version controlled way
• Allows immediate access to source updates.
  • I have added to my personal config submodule.recure = True, the only way to roll
• Provides everyone with an individual copy of GACODE that can be refective of any point in time through the hash

There's more advantages. I think we might have to separate this with a different fork, because OMFIT itself is not dependant on GACODE, But OMFIT-MFE is.

[jcandy]

I would prefer to avoid submodules until I understand why this added complexity is required. We tried submodules in the early days of atom and I think it was a fail. Maybe submodules have improved.

From the OMFIT side, isn't it as simple as setting OMFIT_GACODE_ROOT (and possibly a platform variable)? Conceptually this is all that should be required. I think there is overestimation of the complexity of the "custom GACODE" approach, and an underestimation of the complexity of trying to overautomate the user experience. If there are changes to GACODE required for this maybe that will make GACODE better.

[jcandy]

Imagine that GACODE built only a single executable. Wouldn't the solution be to define a single environment variable that pointed to the executable?

[wdeshazer]

That is what we currently support. That's why I think the submodule is better.
Today, I would characterize OMFIT's utilization as two workflows:

1. GACODE_ROOT points to an institutional load that does not change dynamically.
   • This can be an advantage in some ways, because you aren't subject to rapid changes.
   • It can be a disadvantage, however, when you need those changes to be right.
2. GACODE_ROOT can be changed within the GUI and point to your custom build.
   • This means that you can do the rapid development just as you want (So we already have this workflow. @jmcclena and @avdeevag use this all the time)
   • The disadvantage is, the perservation of the hash is not managed through version control but either custom documentation solutions at best, and word of mouth at worst

With submodules you get that done every time.

One thing we haven't built into omfit is the execution of a build phase. Currently that is managed externally.

As an aside aside. I was talking to @smithsp about this, but we should have a binary repo that keeps all binary traceable to the hash of the source but wouldn't require a rebuild when transmitting to others. That is very down the road, but I like the idea personally.

[jcandy]

After talking with @jmcclena I think the current issue with (2) is module conflicts. The modules in GACODE's platform/env may conflict with the modules loaded by OMFIT. Is that true?

[wdeshazer]

TL; DR We can handle the aspects of pointing to the environment GACODE based environment. What I am excited about is the opportunity to increase traceability. But at some level it will always be on the user to exercise best practices with documentation and traceability or to shoot him/herself in the foot by modifying things in difficult to recover from.

Original content

There will be some changes and we will have to create resources to support a reorganization and we will have to find a budget to pay for this, but I am excited about the prospect. I don't have time to fully flesh things out right now, but I am already re implementing the module design to align with the platform based system based approach.

That said, any and all of the content, whether in GACODE or in a folder at PPPL will never prevent the user from customizing aspects of these choices at runtime. That makes the module in the PLATFORM files a default and so long as someone takes care of what they are doing and understands the risks and the implications, then they can do things like substitute sub-modules within modules.

In OMFIT, we retain run time configuration in the OMFITtree that is saved with the project. Of course is someone changes the settings but doesn't run the run, then we can create disconnects. But the takeaway is that it is only at run time that one knows what the environmental configuration was, assuming decisions don't happen downstream of execution of our requests.

[smithsp]

@jcandy Yes, there can be conflicts between the gacode environment and the OMFIT environment. At one point I tried to bring them in alignment by compiling GACODE with conda tools (OMFIT uses conda exclusively for handling all of its environments and dependency resolution), but that turned out to have a bug for running with more than 32 cores. There is an issue from @bgrierson21 noting this limitation, which he discovered on the PPPL systems. I didn't look further into conda packaging of gacode since then, and have relied on system installations instead for the GACODE environment. Usually the OMFIT modules start with module purge.