Error processing full RAP grid files

[JeffHamiltonNOAA]

Using 10.1.0 utilities, I am getting an unrecognized grid error when trying to process the full RAP rotated lat-lon grid:

MetGrib2DataFile::read_grib2_grid() -> found unrecognized grid definition (32769)

This occurs when I attempt to use GridStat and PCP_Combine on Jet, in the /lfs4/BMC/amb-verif/hamilton directory, with the following commands:

/contrib/met/10.1.0/bin/grid_stat 2209006000006 st4_pr.2022033112.06h.grb2 GridStatConfig
/contrib/met/10.1.0/bin/pcp_combine -name APCP -add 2208912000001 1 test.nc

I'll admit that I have never attempted to use the full RAP grid before, and that the grid itself is a bit strange even for a rotated lat-lon. I should note that the experimental RRFS_NA_3km, also a rotated lat-lon, works just fine with the following command in the same directory:

/contrib/met/10.1.0/bin/grid_stat RRFS_NA_3km.t11z.bgsfcf001.tm00.grib2 st4_pr.2022033112.01h.grb2 GridStatConfig_RRFS_NA

I know that the wgrib2 utility has never agreed with RAP data well, failing to recognize exactly what it is. Perhaps this is a similar problem?

[JohnHalleyGotway]

@JeffHamiltonNOAA, thanks for pointing me to a sample file. I pulled jet:/lfs4/BMC/amb-verif/hamilton/2209006000006 and extracted the 2-m temperature record for testing.

1. wgrib2 doesn't like it:

> wgrib2 2209006000006_TMP_2m.grib2 -V
1:0:vt=2022033112:2 m above ground:6 hour fcst:TMP Temperature [K]:
    ndata=794802:undef=0:mean=279.913:min=233.063:max=302.626
    grid_template=32769:winds(grid):
	I am not an Arakawa E-grid. 
	I am rotated but have no rotation angle.
	I am staggered. What am I?
	(953 x 834) units 1e-06 input WE:SN output WE:SN res 56
	lat0 -10.590603 lat-center 54.000000 dlat 121.813000
	lon0 220.914154 lon-center 254.000000 dlon 121.813000 #points=794802

2. And neither does MET:

> plot_data_plane 2209006000006_TMP_2m.grib2 2209006000006_TMP_2m.ps 'name="TMP"; level="Z2";'
ERROR  : MetGrib2DataFile::read_grib2_grid() -> found unrecognized grid definition (32769)

Glad they're at least consistent!

NCL and Panoply seem to be OK with it.

1. Running ncl_convert2nc and plotting with ncview:

ncl_convert2nc 2209006000006_TMP_2m.grib2
ncview 2209006000006_TMP_2m.nc

2. Panoply makes this plot.

I suspect we're somewhere in a middle ground here. I'm guessing there's some metadata to be improved in those RAP GRIB2 files (so that wgrib2 would be happy), but I'm also guessing that getting them changed isn't a realistic option.

What sort of resolution is NOAA/GSL looking for here?

1. Advice on modifying those GRIB2 files to make wgrib2 (and MET) happy?
2. Modifications to the MET software so that it can read them since that file metadata can't/won't change?
3. Write a python embedding script to read the data and pass it in memory to the MET tools?

[JohnHalleyGotway]

Aha, I understand the situation much better now. Grid definition template 32769 really is a legitimate grid type, as defined here:
https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-32769.shtml

I compared that spec to the rotated lat/lon spec (template = 1):
https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_temp3-1.shtml

While they are similar, there are some significant differences, which explains why I wasn't able to trick the MET software into handling it. Getting MET to process data for this grid template would require an enhancement. It isn't clear to me exactly what's needed. Could we use the 32769 info to instantiate an existing rotated lat/lon grid? Or is an entirely new grid type required?

Based on this info, I conclude that...

1. There is no immediately obvious problem with the metadata in the RAP GRIB2 files.
2. There is no bug in existing MET functionality that needs to be fixed.
3. Adding support for grid template 32769 would be an enhancement that could potentially be added, if needed.

Thoughts?

[JohnHalleyGotway]

@JeffHamiltonNOAA and @TaraJensen, I migrated this info over into a new GitHub issue:
dtcenter/MET#2124

I assigned it to @rgbullock since he'd be a good person to look into the grid details. And @JeffHamiltonNOAA would be a good person to test out any changes.

Note that I put this into the "Consider for next release milestone". So we'll still need to work out when to address this and what project funding to use.