GDAL Example

[anton-seaice]

@mdsumner suggest this morning maybe we could have an example to demonstrate using ACCESS-OM output with geo-spatial analysis tools (i.e GDAL).

The goals would be:

• to encourage us to get the coordinate reference system (CRS) and metadata correct in ACCESS models, so that GDAL can auto-detect this, and
• to demonstrate functionality of GDAL to assist in comparing remote sensing and modeled output.

One possibility would be to compare Sea-Ice Concentration modeled vs observations (similar to https://cosima-recipes.readthedocs.io/en/latest/DocumentedExamples/SeaIce_Obs_Model_Compare.html#Sea-Ice-Concentration-Anomalies). But maybe its a good excuse to look at a different observational dataset. This could be satellite altimetry for sea ice freeboard, although maybe there are other examples (sea surface height?, chlorophyll?, a non-gridded product?, buoy/animal tracks? ).

We would need to investigate the best package to use for this ( GDAL wrappers like rasterio or rioxarray or the gdal python binding seem like the likely candidates). We think rasterio might not support 2D geo-location arrays (as we have for the tripolar grid) but GDAL should. (see discussion)

In ACCESS-OM2 output, its likely some metadata would need to be added manually, but hopefully the notebook would encourage us to get the metadata up to date for OM3.

[anton-seaice]

This example might overlap similar ground to this PROJ example by Aidan

[mdsumner]

I will work through that example as a starting point. I'm still navigating my way on gadi but making progress. I see the temp copy of BedMachine is not in

/g/data/hh5/tmp/aph502/BedMachineAntarctica_2020-07-15_v02.nc

so I'm downloading my own copy (using earthdata creds). But, maybe it's on the thredds? At any rate we can hit it up directly by putting the earthdata creds in GDAL_HTTP_HEADER_FILE, or in GDAL_HTTP_HEADERS - but better to use a shared local copy first so we don't load too much up in the draft example. (I'm not familiar with network access on gadi, I assume we can't just blithely hit up URLs as I've become accustomed to).

[mdsumner]

I got it going enough to the point I could warp cosima output to BedMachine (I realize what I said above was not viable, the other way is not possible, you can't warp to an un-regular grid - the workflow as-is makes sense).

I need some help finding NetCDFs that are relevant, but quick test shows that blithely warping to the Bedmachine grid is fine, and similarly we could use the NSIDC sea ice products in the same way. (I'm interested to flesh out python and R pathways that achieve this, though I've only done cli for now).

[anton-seaice]

I will work through that example as a starting point. I'm still navigating my way on gadi but making progress. I see the temp copy of BedMachine is not in

/g/data/hh5/tmp/aph502/BedMachineAntarctica_2020-07-15_v02.nc

so I'm downloading my own copy (using earthdata creds). But, maybe it's on the thredds? At any rate we can hit it up directly by putting the earthdata creds in GDAL_HTTP_HEADER_FILE, or in GDAL_HTTP_HEADERS - but better to use a shared local copy first so we don't load too much up in the draft example. (I'm not familiar with network access on gadi, I assume we can't just blithely hit up URLs as I've become accustomed to).

urls through the login node or ARE are ok, not through jobs in the the queue though. If they require credentials its probably better to add it to ik11 either way.

[mdsumner]

so I said this up there

way is not possible, you can't warp to an un-regular grid -

it totally is possible, I had no idea. First example here just uses URLs, for the second I downloaded the BedMachine.

https://gist.github.com/mdsumner/380fcc579e7c1736de189fa4e4fa67e9