Percentile thresholding in gen_ens_prod

[hertneky]

I have a use-case that currently runs gen_ens_prod, giving a thresh of >5 m/s for 10-m wind, which is then fed into point stat (WIND_Z10_ENS_FREQ_ge5) to out put linetypes PCT, ECNT, ECLV, etc and producing plots such as Reliability, ROC, ECLV at a given relative frequency or "probability" e.g. == 0.1. This works fine.
I am now considering percentile thresholding in gen_ens_prod to look at, say, a threshold >90th percentile (>SFP90). Looking at gen_ens_prod.cc, I don't see "set_perc_thresh" as I do for other tools, so it appears to not be a capability. I can't think why it wouldn't makes sense to have percentiles in gen_ens_prod. I only see CDP used in there. Let me know if it should be a capability and I am just doing something horribly wrong....thnx!
The error is not clear that this capability is not supported yet, if in fact it isn't "ERROR : Simple_Node::check(double, double, double) const -> percentile threshold ">SFP90" used before it was set." Which since it is not in the code, makes sense that it is not set.

[JohnHalleyGotway]

Tracy, you are exactly correct in your interpretation of that error message:

ERROR : Simple_Node::check(double, double, double) const -> percentile threshold ">SFP90" used before it was set.

That means that the code tried to apply the percentile threshold before the data was used to interpret and set its actual value. And you're also correct to be looking at the set_perc_thresh(...) library function which the MET code must call before those percentile thresholds can actually be used.

Grep-ing through the code, I see it called in the following spots:

.//src/libcode/vx_analysis_util/stat_job.cc
.//src/libcode/vx_shapedata/mode_conf_info.cc
.//src/tools/core/ensemble_stat/ensemble_stat.cc
.//src/tools/core/ensemble_stat/ensemble_stat_conf_info.cc
.//src/tools/core/grid_stat/grid_stat.cc
.//src/tools/core/grid_stat/grid_stat_conf_info.cc
.//src/tools/core/mode/mode_exec.cc
.//src/tools/core/point_stat/point_stat.cc
.//src/tools/core/point_stat/point_stat_conf_info.cc
.//src/tools/core/stat_analysis/stat_analysis_job.cc
.//src/tools/core/wavelet_stat/wavelet_stat.cc
.//src/tools/core/wavelet_stat/wavelet_stat_conf_info.cc

So that means that percentile thresholds can be used in Stat-Analysis, MODE, Ensemble-Stat, Grid-Stat, Point-Stat, and Wavelet-Stat. Noticeably absent from that list are Gen-Ens-Prod, Series-Analysis, and MTD. Search for "Percentile Thresholds" in the config file overview section of the MET User's Guide, and you'll find a similar list of tools, along with how the percentile thresholds are used by those tools. The idea is simple in principle, but the details of exactly what input data to use can get tricky.

Looking in GitHub issues, I see 3 existing issues requesting that percentile thresholding support be added to other tools:

• Consider extending percentile threhsolding to Series-Analysis, Ensemble-Stat, and MTD. MET#1112 for MTD and Series-Analysis
• Add support to Gen-Ens-Prod for computing probabilities using percentile thresholds. MET#1612 for Gen-Ens-Prod
• Enhance Series-Analysis to support the use of percentile thresholds MET#2489 is another one for Series-Analysis

Please feel free to add a comment to dtcenter/MET#1612 to document your interest in this feature, along with any specific implementation suggestions you'd like to recommend. That one is in the "Consider for Next Release" category but is currently not slated for work toward the MET-12.0.0 release. We could loop in @TaraJensen to see if/where it slots into our priorities and whether or not we should work on it prior to 12.0.0.

[hertneky]

@JohnHalleyGotway Thanks for the thorough explanation. Our group was interested in this capability for evaluating our ensemble. Unfortunately, the project is coming to an end at the end of December, so we won't be able to use it for the current year. However, I will still indicate my interest in that Issue, since there is a chance the project will get funded for another year.