Distribution differences
The -m diff and -m ratio metrics can be used to compare the observed and forecasted aggregation statistics. Any aggregator (i.e. mean, median, min, max, std, variance, etc) can be used.
The bias metric (-m bias) shows the aggregated difference between forecast and observations: aggregator(forecast - observation). The statistic difference metric (-m diff) shows the difference in aggregated statistics: aggregator(forecast) - aggregator(observation). When the aggregator is the mean, the two metrics give the same result. However, other aggregators give different results. For example, the following shows the standard deviation of the bias on the left and the difference of the standard deviation on the right:
verif raw.nc cal.nc -m bias -agg std
verif raw.nc cal.nc -m diff -agg std
The right figure shows that the calibrated file has forecasts with variability that is close to the observed, whereas the raw has too little variability.
-m ratio shows the ratio of aggregated statistics, that is aggregator(fcst) / aggregator(obs). To see if the forecasts have the same variance as the observations, use:
verif raw.nc cal.nc -m ratio -agg variance
The frequency diagram (-m freq) shows how often each value is observed and forecasted. -r sets the edges of the intervals to compute frequencies for and -b sets the interval type. For precipitation, a good combination is:
verif raw.nc cal.nc -m freq -r 0,0.00001,1:10 -b =withinsince this creates the bins [0,0.00001), [0.00001,1), [1,2), and so forth. The first bin shows the non-occurrence of precipitation.
