Continuous Ranked Probability Scores (CRPS)

[santiviquez]

Metric's name
CRPS — Continuous Ranked Probability Scores

Metric's category
Regression

Metrics formula

Describe the metrics use cases, and any relevant references.
Continuous Ranked Probability Scores (CRPS)
Continuous Ranked Probability Score (CRPS)
A generalisation of Ranked Probability Score (RPS) is the Continuous Rank Probability Score (CRPSS) where the thresholds are continuous rather than discrete (see Nurmi, 2003; Jollife and Stephenson, 2003; Wilks, 2006). The Continuous Ranked Probability Score (CRPS) is a measure of how good forecasts are in matching observed outcomes. Where:

CRPS = 0 the forecast is wholly accurate;
CRPS = 1 the forecast is wholly inaccurate.
CRPS is calculated by comparing the Cumulative Distribution Functions (CDF) for the forecast against a reference dataset (observations, or analyses, or climatology) over a given period.

References:

• https://arxiv.org/abs/2311.14122
• https://confluence.ecmwf.int/display/FUG/Section+12.B+Statistical+Concepts+-+Probabilistic+Data#Section12.BStatisticalConceptsProbabilisticData-ContinuousRankedProbabilityScore(CRPS)
• https://en.wikipedia.org/wiki/Scoring_rule