As a user, I use the values filter to remove extreme values and would like that to be applied before event detection so that extreme values aren't considered

[HankHerr-NOAA]

I just performed an event detection evaluation for ABRN1 (same data as always) applying this filter:

values:
  minimum: 1.0
  maximum: 10.0

The complete declaration is below; supporting data is in the GitHub_382 WRES Redmine Large Data Sets Google Drive folder. That change did not impact the identified events, but did remove pairs from the evaluation. In other words, the values filter is applied AFTER event detection.

When I talked about this with James, he said that the main use case for values that he foresees is to remove unreasonable numbers (e.g., negative values indicating missing that aren't otherwise recognized as missing) from the time series, so perhaps it should be applied BEFORE event detection.

We can continue the discussion here and, if necessary, implement the change. I'm calling this a feature at this point, not a bug, but am open to negotiations.

Thanks,

Hank

==========

label: Testing Event Based
observed:
  label: OBS Streamflow
  sources: /home/ISED/wres/wresTestData/issue92087/inputs/ABRN1_QME.xml
  variable: QME
  feature_authority: nws lid
  type: observations
  time_scale:
    function: mean
    period: 24
    unit: hours

predicted:
  label: AHPS Streamflow Forecast
  sources:
    - uri: https://WRDS/api/rfc_forecast/v2.0/forecast/streamflow
      interface: wrds ahps
  variable: QR
  type: single valued forecasts

features:
 - {observed: 'ABRN1' , predicted: 'ABRN1' }

time_scale:
  function: mean
  period: 24
  unit: hours

reference_dates:
  minimum: 2000-08-07T23:00:00Z
  maximum: 2015-08-08T23:00:00Z

event_detection: observed

values:
  minimum: 1.0
  maximum: 10.0

metrics:
  - pearson correlation coefficient
  - sample size
  - name: time to peak error
    summary_statistics:
    - median
    - minimum
    - maximum
    - mean

output_formats:
  - csv2
  - png
  - pairs

[james-d-brown]

Thanks, Hank.

On the whole, I think these filters should not be applied before event detection, but I think there is a reasonable case for the values being an exception because the goal of that filter is to completely ignore any unreasonable values from an analysis. More generally, applying filters, such as the season, before event detection probably isn't a good idea as it will introduce a discontinuity into the time-series for event detection, whereas applying the filter post-detection seems reasonable (and is the case now).

Regardless, I think the wiki probably needs a section to explain what filters and transformations are applied before event detection, notably valid_dates, time_scale and, with this ticket, values.

[HankHerr-NOAA]

Makes sense to me. Thanks,

Hank