Feature request: mitgcm_ocean compressed netcdf files for dart

[hkershaw-brown]

Use case

What are you trying to accomplish? Sometimes there may already be a way to do what you need.
Run a high resolution mitgcm ocean case with filter.

Is your feature request related to a problem?

In the released version of DART filter bombs (runs out of memory) when running this case.

Describe your preferred solution

Ed Liu created a 'squished state' solution for his siparcs 2022 project:
774b9cc dart_nc_reduce
This was used to successfully run filter with the 80 member high resolution case.

Updated to use dart utilities, and have the corresponding dart_nc_expand: d197e63

Since mitgcm has an mit_to_dart, dart_to_mit, the logic from the dart_nc_reduce/expand programs could and should be dropped into mit_to_dart, dart_to_mit so we are not doing a double hop
model -> netcdf -> squished netcdf -> filter

Describe any alternatives you have considered

general (not model specific) version. However, we could not drop this into dart at the moment - see issues labeled IO and state_structure. The pressing issue is the high-res mitgcm.

Connected (but ignored for now) non-scaling memory issues :
MITgcm_ocean static data
MPI window copy.

[hkershaw-brown]

https://github.com/NCAR/DART/tree/netcdf-reduce

[hkershaw-brown]

fixed by #640