Design Manifesto

[CJ-Wright]

We might want a design manifesto someplace with things we have learned from developing this software. I don't know where that is and some projects may need their own design manifestos (see redsky) but I am going to put some stuff here for the time being.

Configuration vs. Flexibility

We have run into issues recently with configuration and flexibility. Both xpdAcq and xpdAn have globals (or glbls) which are large dictionaries for holding lots of configuration state. This offers a great amount of convenience to the user as much of the information that they don't want to worry about (where do files get saved when I save them, which database do I talk to) are removed from their control. However, this also causes quite a few headaches. The way that much of this configuration happened inside the code was step 1) import the globals from the global module, step 2) use the globals inside functions and classes when configuration information was needed. This caused the system to be very brittle, if you worked on a new system or needed to change any information you needed to rewrite the globals (in the source code as it was hard coded) and reload the functions.

This was fixed on a couple of levels:

1. Load the globals from a configuration file. This allows for medium term changes to the global state (eg moved to a new computer) to be tracked and configured
2. Separate all functions and classes from configuration issues via args and kwargs. There are now _core functions which have no assumed state in the functions, all state must come in from the args and kwargs (with some defaults for long term configuration). This allows advanced users to completely neglect the globals and us the raw functions (which means that they must provide some of the data gotten from the globals)
3. Configured functions are available using the globals. There is now a module of functions which draw configuration state from the globals via partials. Each function is inspected for its kwargs, if a kwarg matches with a key in the global dictionary then a partial is made of the function with all the applicable global data as pre-loaded kwargs. This means that the users who always were using the globals can continue to use them. Note that this also incentivises the global keys to match function keys.