Possible bug in METcalcpy?

[mollybsmith-noaa]

Hi all,

Not sure if this is a bug or desired behavior, but I've noticed that in METcalcpy, if you're trying to plot bias-corrected RMSE, sometimes the bias correction can make the MSE negative, so that when you take the square root of it you get a NaN back. Is there a method for this stat that applies bias correction after the square root is taken, or are there just going to be cases where BCRMSE is NaN?

[georgemccabe]

@bikegeek , @TatianaBurek , do you have any insight here?

[TatianaBurek]

Here is Tara's answer:
"I'd say at this point no, but we may want to revisit with Eric after this release."
https://dtc-metplus.slack.com/archives/C02P4NVB6HK/p1690922935287339?thread_ts=1690911401.487659&cid=C02P4NVB6HK

[jprestop]

@mollybsmith-noaa Do you have any further questions or would it be ok to mark this Discussion as Answered?

[mollybsmith-noaa]

Thanks for confirming that the function looks correct! The negative values that I've seem so far have been small. calculate_mse and calculate_me do perform rounding operations, so that might be what is leading to this. We need to prevent the errors, though, so what would be the best way to solve this programatically?

[jprestop]

Thank you @ericgilleland and @mollybsmith-noaa. Molly, since this is coming from METcalcpy, I am tagging @fisherhucar and @bikegeek who should be able to assist.

[bikegeek]

@mollybsmith-noaa , in looking at the code, it looks like the calculate_mse() is in sl1l2_statistics.py- is that correct?

Before I remove the rounding, I'll check with Tatiana to see why it is there, perhaps to be consistent with METviewer??? Naively, I could put a check to see if the value is negative and return the non-rounded version. But I'd need to see if removing the rounding actually addresses this issue.

[bikegeek]

Github issue: dtcenter/METcalcpy#329

@mollybsmith-noaa, @ericgilleland et al please feel free to add any other comments to the issue.

[mollybsmith-noaa]

@bikegeek That is the correct module in METcalcpy. The issue is that calculate_bcrmse throws somewhat frequent errors because calculate_bcmse sometimes returns negative numbers.

[mollybsmith-noaa]

I wrote this in the issue for this problem, but I want to be clear that the problem is NOT that calculate_mse and calculate_me are returning negative numbers, they are not doing so. The problem is that the difference between the two can erroneously be negative, which produces an error when trying to take the square root of calculate_bcmse to produce calculate_bcrmse.

[bikegeek]

locking this conversation. Any additional comments should be directed to the Github Issue: dtcenter/METcalcpy#329