RUNTIME_FREQ options behaviour

[mpm-meto]

I want to query the behaviour of the RUNTIME_FREQ options. I am using an INIT_LIST, as shown below and I have a 6-day forecast for each of these dates.

INIT_TIME_FMT = %Y%m%d%H
INIT_LIST=2021032400,2021040700,2021042100,2021050500,2021051900,2021060200,2021061600,2021063000,2021071400,2021072800

LEAD_SEQ = begin_end_incr(24,145,24)

frequency to run the tool valid options include:
RUN_ONCE, RUN_ONCE_PER_INIT_OR_VALID, RUN_ONCE_PER_LEAD, RUN_ONCE_FOR_EACH
GRID_DIAG_RUNTIME_FREQ = RUN_ONCE_PER_INIT_OR_VALID

If I wanted a combined result per lead time for the 10 dates in the INIT list I can't get it with any of the settings. The only cumulative option is if I run with RUN_ONCE_PER_INIT_OR_VALID, and this accumulates all the lead times and all the inits into ONE file and one 2D histogram, which gets the "_all" added to the file name.

RUN_ONCE, RUN_ONCE_PER_LEAD, RUN_ONCE_FOR_EACH all seem to produce the same outcome, and if I specify a non-time/date specific output file, the file is overwritten with every lead time/day so I end up with 6 files, one per lead time, but they only contain results for the last day in the INIT_LIST date sequence. Of course I can add the time/date info into the filename and then add the fields up but sometimes it would be nice to be able to get GridDiag to give that accumulation without doing extra work. I thought that's what the RUNTIME_FREQ options were for. I guess I am wondering whether the intention was that all the options give the same outcome or whether there is some logic error lurking somewhere. I did not expect all of the options to give the same result. I expected RUN_ONCE_PER_LEAD to give me what I wanted.

[mpm-meto]

Very interesting... I have just amended the wrapper config to reflect the date/time string and it has now replaced the values with "all" when running with RUN_ONCE_PER_LEAD. It appears to have produced a single file per lead time which seems to have more than one INIT time in it (based on the size of the counts). I have no idea why making the change to the OUTPUT_TEMPLATE including an init?fmt string should change the behaviour.

[georgemccabe]

I'm not sure that I follow all of the details here, but here is some info that I hope clears up some of the confusion.

1. The files that are found for each of the runtime frequency settings depend on the filename template tags that are used in the input templates. The Runtime Frequency section of the METplus User's Guide contains examples of which files are selected based on the different runtime frequency settings.

If you are trying to process each forecast lead separately using all initialization times specified in INIT_LIST, then GRID_DIAG_RUNTIME_FREQ = RUN_ONCE_PER_LEAD seems like the appropriate setting. However, if the input templates do not contain the appropriate filename template tags, e.g. {lead} and {init} or {valid}, then it may not be able to find the files correctly.

What value do you have set for GRID_DIAG_INPUT_TEMPLATE? This may explain why it isn't finding the correct files. You can also set LOG_LEVEL=DEBUG to see what files it is actually finding.

2. The output template (GRID_DIAG_OUTPUT_TEMPLATE) should be set with consideration of the runtime frequency setting. For example, if you have set GRID_DIAG_RUNTIME_FREQ = RUN_ONCE_PER_LEAD and GRID_DIAG_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H}.nc, then each run will write to all.nc because you are processing multiple initialization times for each run so a single init time cannot be substituted in the output template.

If you are trying to see how the different runtime frequency settings affect the results, I would recommend including both init and lead in the output template.

If GRID_DIAG_OUTPUT_TEMPLATE = init_{init?fmt=%Y%m%d%H}_lead_{lead?fmt=%3H}.nc and:

• GRID_DIAG_RUNTIME_FREQ = RUN_ONCE_PER_LEAD: output will be init_all_lead_{HHH}.nc for each lead
• GRID_DIAG_RUNTIME_FREQ = RUN_ONCE_PER_INIT_OR_VALID: output will be init_{YYMMDDHH}_lead_all.nc for each init
• GRID_DIAG_RUNTIME_FREQ = RUN_ONCE_FOR_EACH: output will be init_{YYMMDDHH}_lead_{HHH}.nc for each init and lead
• GRID_DIAG_RUNTIME_FREQ = RUN_ONCE: output will be init_all_lead_all.nc

[georgemccabe]

Hi @mpm-meto, I am just following up on this discussion. Were you able to resolve the issue or do you need more assistance?

[georgemccabe]

Hi @mpm-meto, just following up again. If I don't hear from you in a week I will close this discussion. Thanks!