valid time period for precip use case

[PerryShafran-NOAA]

Hi, there,

Tagging @LoganDawson-NOAA for awareness of the discussion.

We are working on this particular precip analysis verification use case:
https://metplus.readthedocs.io/en/latest/generated/model_applications/precipitation/PointStat_fcstURMA_obsCOCORAHS_ASCIIprecip.html#sphx-glr-generated-model-applications-precipitation-pointstat-fcsturma-obscocorahs-asciiprecip-py

We are verifying 24 hours worth of analyses precip vs. a 24-hr forecast valid at 12Z from CoCoRaHS. We have the following entries into the configuration file:

LOOP_BY = VALID
VALID_TIME_FMT = %Y%m%d%H
VALID_BEG = {ENV[VDATE]}12
VALID_END = {ENV[VDATE]}12
VALID_INCREMENT = 1M

OBS_POINT_STAT_WINDOW_BEGIN = -82800
OBS_POINT_STAT_WINDOW_END = 3600

If you have WCOSS2 access, it's at /lfs/h2/emc/vpppg/noscrub/perry.shafran/EVS_analyses/parm/metplus_config/analyses/precip/stats/PointStat_fcstANALYSES_obsCOCORAHS_ASCIIprecip.conf

But we noticed in the resulting stat file the following oddities about the times:

FCST_VALID_BEG/END - 20230604_120000 / 20230604_120000
OBS_VALID_BEG/END - 20230603_130000 / 20230604_130000

Looking at the log file, we believed that the correct 24 hours worth of analyses were compared with the correct CoCoRaHS file. But these labels seemed to be a weird discrepancy. Are our statistics all right do you think? We can't find where these values were generated.

Thanks!

Perry

[JohnHalleyGotway]

Hi @PerryShafran-NOAA, I see you have some questions about your METplus Use Case for verifying 24-hour precipitation against CoCoRaHS point observations.

To put it simply, the forecast and observation valid times you're seeing in the output are totally consistent with the configuration settings you describe. Let me explain why.

1. Each time you run Point-Stat and Grid-Stat, it is always the case that FCST_VALID_BEG = FCST_VALID_END. Any single forecast field you evaluate has one, and only one, valid time. So the timestamps in those two columns will always be the same.
2. This is not the case for Stat-Analysis, for example. Stat-Analysis jobs read multiple lines of .stat output and keep track of the minimum and maximum valid times encountered. The minimum is reported in the FCST_VALID_BEG output column and the maximum is reported in the FCST_VALID_END output column.
3. Each time you run Grid-Stat, OBS_VALID_BEG = OBS_VALID_END for the same reason described in 1. However, for Point-Stat, OBS_VALID_BEG and OBS_VALID_END will generally differ and are based on the observation window begin/end configuration settings. Those columns define the observation "retention time window" used for that run.
4. That point observation retention time window is computed as [ forecast valid time + window begin, forecast valid time + window end]. In your example, that's [ 20230604_120000 + (-82800 seconds), 20230604_120000 + 3600 seconds ]. And that results in:

OBS_VALID_BEG = 20230603_130000 
OBS_VALID_END = 20230604_130000

I'd recommend looking very carefully at the definition of the 24-hour forecast accumulation you're verifying. Does it really go from 13Z on the previous day to 13Z on the current day? If so, all is fine. Just confirm that it shouldn't be 12Z to 12Z instead (which is what my first guess would be).

My guess is that the question/confusion is related to the FCST_VALID_BEG and FCST_VALID_END settings. Perhaps you're expecting that they'd indicate the accumulation time interval? To clarify, they do not. Instead, MET typically appends the accumulation interval to the output variable name. So you'll see APCP_24 in the output to indicate a 24-hour accumulation of precip instead of just APCP.

Please let me know what other questions you have.

Thanks,
John

[PerryShafran-NOAA]

@JohnHalleyGotway @LoganDawson-NOAA

Thanks for your response. StatAnalysis is not part of this particular use case. This makes sense. We were just wondering the explanation for these supposed inconsistencies. We were just asking why the forecast valid times did not seem to match the obs valid times.

Keep in mind that in this case we are accumulating 24 1-hr analyses, not a forecast accumulation. I don't recall who was the original contact for this particular use case, but it was definitely done with the CoCoRaHS in mind.

Perry

[LoganDawson-NOAA]

Thanks for the information @JohnHalleyGotway

@PerryShafran-NOAA I had not noticed the definitions of OBS_POINT_STAT_WINDOW_BEGIN/END in the original use case. I now see the intention behind using those window settings (-23 h / +1 h) to verify accumulations with a valid time of 23Z. The CoCoRaHS data that we're reading are daily accumulations that are typically recorded around 7 AM local time, so this windowing isn't really appropriate.

What would be more appropriate for our application of this use case would be to set OBS_POINT_STAT_WINDOW_BEGIN/END = -/+600 to allow for a -/+10 min window about 12Z for verifying the 12Z-12Z QPE analyses.

In the long run, we should add separate runs of this use case for valid times of 11Z, 12Z, 13Z, 14Z, and 15Z where we verify 24-h QPE accumulations over 11Z-11Z, 12Z-12Z, 13Z-13Z, 14Z-14Z, and 15Z-15Z time frames. That approach would account for CoCoRaHS observations being reported at 7 AM local time, which only falls at 12Z for the Eastern and Central time zones depending on whether we're in Daylight Savings Time or not.

[JohnHalleyGotway]

@LoganDawson-NOAA and @PerryShafran-NOAA wanted to follow up on this old discussion from a few months ago. Can we mark it as being answered, or is more discussion needed?

[PerryShafran-NOAA]

Logan is no longer with us. We'll have to punt on this for now, as analysis precip will not be in EVSv1, but will be in dev and then in EVSv2, so we're going to have to do this right. I haven't got any time to deal with this at the moment.