Taylor Diagram gives different results based on data order #2571
Replies: 5 comments
-
|
We need to see if the files are actually identical, just in a different order and if so, figure out what data the Taylor Diagram is plotting |
Beta Was this translation helpful? Give feedback.
-
|
@bikegeek @fisherhucar There is no orderly output on any CNT linetype to 32 metdataio rows. Plus there are 33119 records. That make it difficult to compare. I do not know how to compare them manually or other ways. Do you need original daily PointStat with CNT linetype to start with? Please advise. Ho-Chun |
Beta Was this translation helpful? Give feedback.
-
|
I put the daily sample PointStat output here |
Beta Was this translation helpful? Give feedback.
-
|
Thanks Ho-Chun we think that the two files that you included in the Users Meeting notes have different values as well as different order. Below is how we tested it. Would you double check to see if the two sets of data are the same on your end? We did a wget to grab the files and then pulled out just one time step for one region to compare the values wget https://www.emc.ncep.noaa.gov/mmb/hchuang/ftp/pm25_20240418_20240422_time_evscnt.txt egrep '10000 2024-04-20 07:00:00 2024-04-20 07:00:00 2024-04-20 06:00:00' pm25_20240418_20240422_time_evscnt.txt | grep GreatBasin | grep RMSE | sort > pm25.GreatBasin.RMSE.txt 9945 V11.0.2 pm25_bc NA 10000 2024-04-20 07:00:00 2024-04-20 07:00:00 2024-04-20 06:00:00 0 2024-04-20 06:00:00 2024-04-20 06:00:00 PMTF 10^-6g/m^3 L1 PM25 NA A1 AIRNOW_HOURLY_AQOBS GreatBasin BILINNA NA -9999 0.05 CNT 96 RMSE 6.20562 NA NA NA NA grep '10000 2024-04-18 07:00:00 2024-04-18 07:00:00' evs.taylor.aqm.atmos.grid2obs_pm25.20240418_20240422.txt | grep GreatBasin | grep RMSE | sort > evs.GreatBasin.RMSE.txt 9 V11.0.2 pm25_bc NA 10000 2024-04-18 07:00:00 2024-04-18 07:00:00 2024-04-18 06:00:00 0 2024-04-18 06:00:00 2024-04-18 06:00:00 PMTF 10^-6g/m^3 L1 PM25 NA A1 AIRNOW_HOURLY_AQOBS GreatBasin BILINNA NA -9999 0.05 CNT 92 RMSE 4.01125 NA NA NA NA |
Beta Was this translation helpful? Give feedback.
-
|
@fisherhucar Hank, you compare the wrong metdataio record, fcst_valid_beg, 2024-04-20 vs 2024-04-18. (a) pm25_20240418_20240422_time_evscnt.txt and (b) evs.taylor.aqm.atmos.grid2obs_pm25.20240418_20240422.txt |
Beta Was this translation helpful? Give feedback.
-
Ho-Chun: Different Taylor Diagram between n-days metdataio conversion versus the grouping of 1-day metdataio_converted_stats.
Follow the instruction to create Taylor Diagram using METPlotpy;
Output daily CNT point_stat stats (pm25_raw and pm25_bc)
Copy n-days of point_stat stats to a staging_dir (rename MODE=AQM to pm25_raw and pm25_bc)
Use metdataio/metreformat to convert all point_stat in the staging_dir to a single ascii text file.
Use the ascii text file of (3) as the input for METPlotpy to produce Taylor Diagram
It is time consuming of step (3), 30-day metdataio conversion on the PointStats that has only CNT linetype is (estimated) about 43 mins. (10:07 mins per 7 days of conversion; 4 variables and two models)
Alternative approach;
Output daily CNT point_stat stats
Perform daily metdataio conversion (wallclock time 00:01:36; pm25_raw+pm25_bc) to generate daily ascii text file
Combine n-day of metdataio conversion ascii text files into a single ascii text file (remove header and adjust index number; wallclock time = 00:01:02, delimiter is Tab mark)
Use ascii text file of (C) as the input for METPlotpy to produce Taylor Diagram
Examples shown are from 04/18-04/22 2024. The Taylor Diagram is different between (4) and (D). It is difficult to compare the single ascii text file between (3) and (C). The output order of 32 new Stats_Name from one CNT linetype is random from one CNT conversion to another CNT conversion. Spot check of a few selected records (of the same CNT) showed they have the same values. I need DTC’s help to figure out the cause of the different Taylor Diagram.
Beta Was this translation helpful? Give feedback.
All reactions