wave-ice coupling decreases runtime

[NickSzapiro-NOAA]

A curious result of switching wave-ice coupling on (in ufs-weather-model regression testing) is that the wave model is noticeably ~2x faster for otherwise same configuration. This is consistent on Hera and Derecho.

Example summary ESMF profiles for 14 d coupled simulations are attached for wavice_off.ESMF_Profile.summary and wavice_on.ESMF_Profile.summary, particularly

Region                                        PETs   PEs    Count    Mean (s)    Min (s)     Min PET    Max (s)   Max PET
[WAV] RunPhase1        [wave-ice off]          60     60     600      2592.5817   1190.9005   183     3799.0977   212
[WAV] RunPhase1        [wave-ice on]           60     60     600      1355.6936   908.8005    209     2129.2927   237

Note that the ICE RunPhase takes longer with wave-ice coupling (418 s vs. 327 s), which is what I expected for waves although perhaps not as substantially.

This ufs-weather-model branch can be used to reproduce by running rt.sh -a ACCT -e -k -l my_rt.conf where my_rt.conf is

 COMPILE | s2sw_pdlib_wi | intel | -DAPP=S2SW -DCCPP_SUITES=FV3_GFS_v17_coupled_p8 -DPDLIB=ON -DWAVICE=ON | - noaacloud | fv3 |
 RUN | cpld_control_pdlib_p8_wi                             | - noaacloud                | baseline |
 
 COMPILE | s2sw_pdlib | intel | -DAPP=S2SW -DCCPP_SUITES=FV3_GFS_v17_coupled_p8 -DPDLIB=ON | - noaacloud | fv3 |
 RUN | cpld_control_pdlib_p8                                | - noaacloud                | baseline | 

Especially to evaluate the performance of inner vs. outer loop run sequence and enabling wave-ice coupling, we would like to understand this better. Any ideas/suggestions are most welcome. A caveat is that I don't know how much has been done to load balance these regression testing configurations @JessicaMeixner-NOAA @DeniseWorthen @junwang-noaa

[JessicaMeixner-NOAA]

Are any of your settings in the ww3_grid file different with and without coupling?

For evaluation of inner vs outer loop coupling, I've found the low resolution meshes in the ufs-weather-model are insufficient to get a good idea on impact. I do not know how much the regression tests have been load-balanced.

At the moment I do not know of any one in the wave team who has availability to dig into this further unfortunately due to other high priority tasks.

[NickSzapiro-NOAA]

Thanks @JessicaMeixner-NOAA! I forgot that we changed the number of wave frequencies and directions...

diff ww3_grid.inp.exp.global_270k  ww3_grid.inp.ic4.exp.global_270k
15c15
<  1.07 0.035 33 36 0.
---
>  1.07 0.035 25 24 0.
165a166,167
> $wave ice coupling, ic4 method
> &SIC4 IC4METHOD = 10 /