Vector Statistics and Rotated Lat/Lon grid compatibility

[heu3becteh]

Are Vector Statistics compatible with a Rotated Lat/Lon grid?

I did a check and the forecast values are written to the MPR as is, without rotating the U/V components.
There is also a discussion regarding converting WRF data from the native grid, so I'm guessing the conversion must be done outside MET, even though the Rotated Lat/Lon grid is marked as compatible here (it is in fact supported for scalar fields, and the nearest nodes are found correctly, the problem is with wind direction)?

Is there any way to correctly calculate Vector Statistics for a Rotated Lat/Lon grid?

Best regards,
Alexander

[JohnHalleyGotway]

@heu3becteh apologies for the delay in responding to this one. I see you have a question about the handling of rotating wind vector components from being defined as grid-relative to earth-relative. Let me start by saying that MET does include logic for rotating U/V wind components, but it certainly is possible that it isn't working the way we want for your data. This rotate_uv_grid_to_earth(...) utility function handles it.

First, we assume that the U/V wind components in point observations are defined as being earth-relative... meaning they're defined relative to the poles of the earth. Vector winds in gridded model output can be defined either as being earth-relative or grid-relative, meaning they're defined based on the orientation of the grid. And the grid orientation can differ considerably from true north.

When reading wind components from GRIB1 or GRIB2 files, MET checks the metadata and automatically rotates them when needed. This MetGrib2DataFile::check_uv_rotation(...) function handles it for GRIB2. The 5th bit in the GRIB2 Resolution and Component Flags indicates whether winds are defined as grid or earth-relative. All grid-relative vectors are rotated to be earth-relative automatically.

When processing GRIB1 or GRIB2 input data, the following Debug(3) log message indicates that wind rotation is occurring:

Rotating U and V wind components from grid-relative to earth-relative.

However, based on the content of your question, I'm guessing you're using NetCDF input data, not GRIB. And grep-ing through the MET codebase, I see that the rotate_uv_grid_to_earth(...) utility function is only called in the GRIB1 and GRIB2 libraries. So MET isn't currently doing any wind rotation for NetCDF inputs.

This is related to issue dtcenter/MET#1518, to generalize MET's support for the automated derivation of wind speed and direction. Again, automatic derivation of wind speed and direction is also currently only supported for GRIB1 or GRIB2 inputs. It seems to me that that issue should be expanded to include wind rotation as well. Ideally, we'd like for MET to be able to read U/V input or Wind Speed/Direction inputs, automatically rotate them when needed, and then verify them against observations of U/V wind or Wind Speed/Direction. But work is needed to add that level of flexibility.

I'm also not aware of any NetCDF convention that defines whether or not the rotation is necessary. Do you know of anything that indicates HOW the vector components are defined, similar to the resolution and components flag for GRIB? Lacking that, I suppose we could choose some default behavior (e.g. automatically rotate or not), and then provide a configuration option to turn the rotation on/off as needed.

[RogerHar]

Hi John,
(Butting in as I happen to be online and got an email alert for your comment on #1518): The CF metadata conventions for NetCDF variable names are that x_wind and y_wind are relative to the model grid, while eastward_wind and northward_wind are relative to the earth. They're equivalent for a lat-lon grid obviously, and current Met Office global model output on a lat-lon grid uses x_wind and y_wind. See the CF Standard Name Table. (It lists grid_eastward_wind and grid_northward_wind as aliases for x_wind and y_wind - they strike me as rather confusing and I don't know if anyone uses them but I guess it would be good to support them in case.)

[heu3becteh]

@JohnHalleyGotway Thank you for such a detailed and useful reply!

When reading wind components from GRIB1 or GRIB2 files, MET checks the metadata and automatically rotates them when needed. This MetGrib2DataFile::check_uv_rotation(...) function handles it for GRIB2. The 5th bit in the GRIB2 Resolution and Component Flags indicates whether winds are defined as grid or earth-relative. All grid-relative vectors are rotated to be earth-relative automatically.

In my case resolutionAndComponentFlags define data as uvRelativeToGrid:

119-120 codeflag resolutionAndComponentFlags = 56 [00111000:(3=1)  i direction increments given ;(4=1)  j direction increments given ;(5=1)  Resolved u- and v- components of vector quantities relative to the defined grid in the direction of increasing x and y (or i and j) coordinates, respectively :grib2/tables/19/3.3.table]
120-120 bit uvRelativeToGrid = 1

But the following message is missing in the log (with -v 8):

When processing GRIB1 or GRIB2 input data, the following Debug(3) log message indicates that wind rotation is occurring:

Rotating U and V wind components from grid-relative to earth-relative.

And the value MET writes as FCST in MPR data is the same I get simply as nearest node using pygrib codes_grib_find_nearest.
To be honest, I can't rule out the possibility that the model output earth-relative wind, but GRIB2-metadata indicates otherwise, wgrib2 -vector_dir also gives winds(grid).

I'm also not aware of any NetCDF convention that defines whether or not the rotation is necessary. Do you know of anything that indicates HOW the vector components are defined, similar to the resolution and components flag for GRIB? Lacking that, I suppose we could choose some default behavior (e.g. automatically rotate or not), and then provide a configuration option to turn the rotation on/off as needed.

In addition to what @RogerHar said, in case of COSMO-model it is U/U_10M:standard_name = "grid_eastward_wind", V/V_10M:standard_name = "grid_northward_wind".

[JohnHalleyGotway]

@heu3becteh if you'd like to pass along a sample file, I could take a closer look at what's going on with the wind rotation logic. Please see these instructions for How to Send Us Data. And please respond to this discussion if/when you post data to our anonymous FTP site so that I know to go grab it.

[heu3becteh]

@JohnHalleyGotway Thank you, would you kindly look at this file:
/incoming/irap/met_help/heu3becteh_data/exampleUVgrid.grib2

[JohnHalleyGotway]

@heu3becteh, my apologies, it looks like that file already aged off of our anonymous ftp site before I pulled it down. Are you able to repost it there?

[heu3becteh]

@JohnHalleyGotway Done.

[JohnHalleyGotway]

Thanks for posting that exampleUVgrid.grib2 data file. I am seeing the same behavior you describe - that MET is NOT rotating the winds from grid to earth relative for this data.

Here's the short answer. I found a bug on this line of code.

rec->ResCompFlag  = gfld->igdtmpl[ 0 == gfld->igdtnum ? 13 : 11 ];

Should be changed to:

rec->ResCompFlag  = gfld->igdtmpl[ ( 0 == gfld->igdtnum || 1 == gfld->igdtnum ) ? 13 : 11 ];

I'll write up an issue and create a bugfix to address this.

If you're interested, here's the steps I used to debug this behavior:

I tested this using the plot_data_plane tool and inspecting the log output.

1. With the example data you sent:

plot_data_plane exampleUVgrid.grib2 rotll_ugrd.ps 'name="UGRD";level="Z10";' -v 3

2. With another GRIB2 file on a lambert conformal projection:

plot_data_plane lc_grid.grib2 lc_grid_ugrd.ps 'name="UGRD";level="Z10";' -v 3
...
DEBUG 3: MetGrib2DataFile::check_uv_rotation() -> Found pair match "VGRD/Z10" in GRIB2 record 224 field 1
DEBUG 3: Rotating U and V wind components from grid-relative to earth-relative.

Those DEBUG 3 log messages indicate that wind rotation is occurring, and those are NOT logged for the data you sent.

However, comparing the wgrib2 output for UGRD, both of these files seem to indicate that the winds are grid relative:

wgrib2 -V exampleUVgrid.grib2
    grid_template=1:winds(grid):
wgrib2 -d 223 -V lc_grid.grib2
    grid_template=30:winds(grid):

The exampleUVgrid.grib2 file uses grid template 1 with the Resolution and Component flags in Section 3, Octet 55:

wgrib2 -get_byte 3 55 1 exampleUVgrid.grib2 
1:0:3-55=56

The lc_grid.grib2 file uses grid template 30 with the Resolution and Component flags in Section 3, Octet 47:

wgrib2 -d 223 -get_byte 3 47 1 lc_grid.grib2 
223:80617180:3-47=56

Both have a value of 56 which translates into bits 00111000 (from most to least significant bit).
So that's all good.

I updated the source code to print out the Resolution and Component Flag stored by MET and found:

ResCompFlag = 56 # For lc_grid.grib2 
ResCompFlag = -30000000 # exampleUVgrid.grib2

This indicates that there's a bug in how the MET library code is parsing the Resolution and Component Flag for this data.

[JohnHalleyGotway]

FYI, here's the MET issue to fix this bug: dtcenter/MET#2711

[heu3becteh]

Thank you! I will test this fix and report the result.

[JohnHalleyGotway]

I've been looking more into this and have also found an additional issue on this line of code. The rotated lat/lon grid code is returning a rotation angle of 0, meaning that no rotation actually occurs.

I need to work some more on this.

No need for you to test at this point because you still won't see any difference in the result.