Deployment architecture, etc

[lars-t-hansen]

Some preliminary notes, to be fleshed out.

For a typical cluster we'll have:

• one sonar per node for sampling
• one sonar per node for sysinfo
• at least one sonar per cluster for slurm data, but probably more than one for redundancy - depends on where, how often
• some kind of exfiltration method
• a remote database for storage
• maybe an on-cluster broker for message forwarding, TBD

There are various scripts for these things currently kept in the Jobanalyzer repo but that's not a great place for them. It's unclear if Jobanalyzer can handle all the redundancy of slurm data that would result from having the slurm agent run on multiple nodes. There isn't any documentation.