Output files when running ugrid data

[robdarvell]

We've tested out some LFRic data using MET12.0.0 for both Gridstat and Pointstat. Both appear to run through fine.
However, we are not getting any forecast range information in either the filename or within the output files, it is always being set to 0. The time information is looping as we expected in the files, but it isn't updating the forecasts accordingly.
eg: Running from a 00Z on 1st Sep forecast file
point_stat_000000L_20230901_060000V.stat
point_stat_000000L_20230901_120000V.stat
point_stat_000000L_20230901_180000V.stat
point_stat_000000L_20230902_000000V.stat
point_stat_000000L_20230902_060000V.stat
point_stat_000000L_20230902_120000V.stat
point_stat_000000L_20230902_180000V.stat
point_stat_000000L_20230903_000000V.stat
point_stat_000000L_20230903_060000V.stat
point_stat_000000L_20230903_120000V.stat

I did try and examine the stat analysis use case to see if that gave me an indications as to what we are doing wrong, but I see that this is only set up for a T+0 example.
Any ideas?

[JohnHalleyGotway]

@willmayfield, @DanielAdriaansen, and @hsoh-u, hoping to loop you two in on this discussion with @robdarvell about reading LFRic data (unstructured grid) into Point-Stat and Grid-Stat. While data is read and output is written, Rob's asking about the metadata - specifically the forecast lead time.

I was hoping to run an LFRic example to see if/how the UGRID metadata is handled, but couldn't find one.

I did some hunting around and found the following details...

• Issue Add support for NetCDF files following the UGRID convention MET#2231 described adding UGRID support for MET version 12.0.0.
• The corresponding pull request Feature 2231 unstructured grid MET#2723 added this support.
• UGridConfig_lfric does provide a sample config file for reading LFRic data, but this not exercised by any of the MET unit test XML's. There are references to MPAS in unit_ugrid.xml but nothing for LFRic.

Do you have any working examples that I could use for debugging?

[JohnHalleyGotway]

@hsoh-u, checking in ugrid_file.cc, I do see that InitTime is a member of the UGridFile class, which is good.

However, it does not ever seem to be set! And so, the vx_data2d_ugrid library code just sets the init time to the same value as valid time, as seen starting on this line:

  //  if unset, set the init time to the valid time
  unixtime init_ut = InitTime;
  if(init_ut == 0 && valid_ut != 0) {
     mlog << Debug(4) << "UGridFile::getData() -> "
          << "setting the unset init time to the valid time of "
          << unix_to_yyyymmdd_hhmmss(valid_ut) << ".\n";
     init_ut = valid_ut;
  }

That would explain the behavior that Rob's seeing.

@hsoh-u I think we'll need a new MET bugfix issue to correct the metadata handling for LFRic data, setting InitTime = forecast_reference_time from the input file. And we should add a unit test to demonstrate. @j-opatz, tagging you here, to loop you into this discussion.

@robdarvell, I'm not sure if this will actually work or not, but a you could try manually defining these timestamps using the set_attr_init, set_attr_valid, and/or set_attr_lead configuration options described in this section.

These options certainly work to override the metadata from other gridded input files. I have just never tested them for UGRID inputs. Please let me know if they work!

[DanielAdriaansen]

Hi @JohnHalleyGotway. Indeed, this is the behavior that I observed when testing in April 2024, but failed to notice the output filename was improperly named. This was an oversight on my part during testing. For a LEAD_SEQ=18 here were my output files from some LFRic data:

-rw-r--r-- 1 dadriaan rap  18K Apr 24  2024 point_stat_000000L_20210728_180000V_mpr.txt
-rw-r--r-- 1 dadriaan rap  16K Apr 24  2024 point_stat_000000L_20210728_180000V.stat

The FCST_LEAD column also looks incorrect in the mpr file.

I have everything set up to test any fixes on Seneca, from my testing back in April 2024. I'll coordinate with you and anyone else via Slack.

[robdarvell]

Hi All,
Thanks for the updates. Just to let you know that adding the set_attr_lead within the config file does set values into the filename and within the output results files when using the LFric data.

[DanielAdriaansen]

@robdarvell thanks for updating us that set_attr_lead worked. We've opened a bugfix issue for this here: dtcenter/MET#3056. I am going to close this discussion and ask that any new conversation be moved to that issue. Thanks!