pgb2file_f024_24_0p25: No such file or directory #3069
Replies: 2 comments 6 replies
-
|
I suspect this is an instance of regenerating/rerunning an experiment after a failed forecast. The forecast generates restart files in the I suggest starting fresh by deleting On a side note, I think we should add a feature to the |
Beta Was this translation helpful? Give feedback.
-
|
Hi @DavidHuber-NOAA - good to know! This brings up a bit of a general question - as far as I can tell, the model runs all completed successfully, but I did have multiple failed atmos_prod jobs as I did some debugging. This was just a 48h test run, so rerunning everything clean isn't much an issue, but in the future I will be making much longer runs that will be expensive to repeat. Is there a way to simply remove the files that atmos_prod is choking on without rerunning the entire experiment? |
Beta Was this translation helpful? Give feedback.
-
|
@benjamin-cash Unfortunately not. The issue is that the forecast model starts at a later forecast hour (if it finds a restart file at 36 hours, that's where it will start). But if you restarted the experiment, then the COMROOT directory will not contain the forecast products for hours 0-33, which is what causes the |
Beta Was this translation helpful? Give feedback.
-
|
Oof. Well, not the answer I wanted, but much appreciated so that I don't spend time banging my head against this - many thanks. :) |
Beta Was this translation helpful? Give feedback.
-
|
Actually, one more question for you. In my case, I didn't realize that this was an issue until the forecast ensemble had completed and a large number of prod tasks had been scheduled. Is there an efficient way to rewind rocoto to the forecast tasks, or do I literally have to go through and rewind every individual prod task? Somewhat baffling, not even deleting the .db file has caused rocoto to reset to the beginning. I would really like to avoid having to create a new experiment directory and make all of the necessary changes to my .xml file again if I can avoid it... |
Beta Was this translation helpful? Give feedback.
-
|
Yes, you can rewind by targeting the forecast metatask. The metatasks are still grouped by member, but at least you don't have to rewind all of the segments. You can probably do it in a loop as well. Something like num_members= # set the member count
cycle=1994050100
for mem in $(seq 0 $num_mems); do
mem3=$(printf %03d $mem)
metatask="gefs_fcst_mem${mem3}"
rocotorewind -w C192mx025_nonans.xml -d C192mx025_nonans.db -c $cycle -m $metatask
done |
Beta Was this translation helpful? Give feedback.
-
|
Hi @DavidHuber-NOAA - I made a new clean run where the fcst task ran and exited without issue, and f000 ran without issue for atmos_prod but f024 crashed with the same error. I've attached the various log files here in case something else suggests itself. |
Beta Was this translation helpful? Give feedback.
-
|
@aerorahul It looks like the offending issue is noted on line 7637 of atmos_prod_mem002_f024.log: Later, the job fails because it cannot find |
Beta Was this translation helpful? Give feedback.
-
I am hitting an error in atmos_prod where it lists the final hour (so f024_24, or f048_48) as no such file or directory. I'm afraid I'm not familiar enough with the prod tasks to understand where this is going wrong and was hoping someone could help me out. The log file from one of the failed tasks is attached here. Thanks!
atmos_prod_mem001_f024.log
Beta Was this translation helpful? Give feedback.
All reactions