Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds a broadcast of the namelist value 'urbantvmapalgo', also fixed stream options so it uses NUOPC options rather than MCT #2534

Open
wants to merge 65 commits into
base: b4b-dev
Choose a base branch
from
Open
Changes from 16 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
bdf6b32
Changes to remove 0th timestep
olyson Jun 1, 2023
a9eb1e7
Changes for lilac
olyson Jul 25, 2023
b8be204
Remove do while loop and dosend variable.
olyson Jul 25, 2023
7f38c07
Fix unit tests for nstep starting at 1 instead of 0
billsacks Jul 25, 2023
55184f4
Merge pull request #1 from billsacks/zerothtstep
olyson Jul 26, 2023
750ed1d
Cleanup.
olyson Jul 27, 2023
198ac31
Merge branch 'zerothtstep' of https://github.com/olyson/ctsm into zer…
olyson Jul 27, 2023
caa384b
Remove comment.
olyson Aug 1, 2023
0dfd901
Remove BACKWARD COMPATIBILITY comment.
olyson Aug 3, 2023
d5b007f
Merge tag 'ctsm5.1.dev147' into zerothtstep
olyson Nov 1, 2023
c9c89a8
Correct a comment.
samsrabin Nov 1, 2023
d059220
Print error()s to console, not just log file.
samsrabin Nov 1, 2023
b311d1d
Rearrange search for h2 files in import_and_process_1yr().
samsrabin Nov 1, 2023
fa843e5
Fix h2 file patterns in import_and_process_1yr().
samsrabin Nov 1, 2023
11b6042
Merge pull request #2 from samsrabin/zerothtstep-fix-RXCROPMATURITY
olyson Nov 2, 2023
258792e
If avgflag /= 'I', time_bounds is present and time = mid of time_bounds
slevis-lmwg Mar 28, 2024
bf059a4
Adds a broadcast of the namelist value (urbantvmapalgo) after reading…
briandobbins May 9, 2024
1738156
Merge remote-tracking branch 'escomp/b4b-dev' into fix_urbantvmapalgo…
samsrabin Oct 8, 2024
020db62
Merge commit '258792e9c2ecf34d5e7dd58b556b34330edd0884' into hist_tim…
slevis-lmwg Oct 17, 2024
1e81456
Remove a comment unrelated to these modifications
slevis-lmwg Oct 17, 2024
268ad5a
Merge tag 'ctsm5.3.010' into zerothtstep
slevis-lmwg Nov 12, 2024
d07fd9d
Removed comments with prefix !KO
slevis-lmwg Nov 12, 2024
0f54bc2
Merge tag 'ctsm5.3.011' into zerothtstep
slevis-lmwg Nov 12, 2024
a4b9dcf
Update docstring of is_first_step function
slevis-lmwg Nov 12, 2024
4d0ac48
New update of function's docstring
slevis-lmwg Nov 12, 2024
e578ae7
Fix typo in comment
slevis-lmwg Nov 12, 2024
bd51b8c
Correct a variable name that came in with a merge conflict
slevis-lmwg Nov 12, 2024
e87b119
Correct another error that came in with a merge conflict
slevis-lmwg Nov 13, 2024
18acd55
Merge remote-tracking branch 'escomp/master' into hist_time_mid_of_ti…
slevis-lmwg Nov 13, 2024
c42cd12
Update .gitmodules to the mosart/rtm tags corresponding to this ctsm
slevis-lmwg Nov 13, 2024
71fbe09
cropcal_module.py import_output(): Handle "instantaneous files."
samsrabin Nov 14, 2024
a51816e
Reformat with black.
samsrabin Nov 15, 2024
3220dbb
Add previous commit to .git-blame-ignore-revs.
samsrabin Nov 15, 2024
760eb96
Merge pull request #16 from samsrabin/ctsm_hist_time_mid_of_time_boun…
slevis-lmwg Nov 15, 2024
ea91981
Draft ChangeLog/ChangeSum
slevis-lmwg Nov 15, 2024
c7e1366
Revert change I brought in with the conflicts
slevis-lmwg Nov 15, 2024
f85104b
Merge branch 'hist_time_mid_of_time_bounds' into zerothtstep
slevis-lmwg Nov 15, 2024
5148dbc
Update .gitmodules to the mosart/rtm tags corresponding to this ctsm
slevis-lmwg Nov 15, 2024
dc295f7
Correct variable name that came in with the conflicts
slevis-lmwg Nov 15, 2024
eeedbc6
Change "if instantaneous" statement with more appropriate conditional
slevis-lmwg Nov 22, 2024
957d032
Merge branch 'b4b-dev' into fix_urbantvmapalgo_broadcast
ekluzek Dec 4, 2024
ffeb713
Merge tag 'ctsm5.3.017' into hist_time_mid_of_time_bounds
slevis-lmwg Jan 9, 2025
332f7c1
Update to mosart1.1.06 and rtm1_0_84
slevis-lmwg Jan 10, 2025
5d87682
Update ChangeLog/Sum
slevis-lmwg Jan 10, 2025
3ffe654
Finalize ChangeLog
slevis-lmwg Jan 11, 2025
fb89f33
Merge pull request #2838 from slevis-lmwg/hist_time_mid_of_time_bounds
slevis-lmwg Jan 11, 2025
8b2f216
Merge remote-tracking branch 'escomp/master' into zerothtstep
slevis-lmwg Jan 11, 2025
f3a1dc6
call advance_timestep if startup or hybrid, not branch or continue
slevis-lmwg Jan 13, 2025
b66e1e9
Draft ChangeLog/Sum files
slevis-lmwg Jan 14, 2025
8d198cf
Update ChangeLog
slevis-lmwg Jan 14, 2025
71e56b9
Update ChangeLog
slevis-lmwg Jan 14, 2025
2f6b129
Merge pull request #2084 from olyson/zerothtstep
slevis-lmwg Jan 14, 2025
6d1a0e0
[BROKEN] Merge tag 'ctsm5.3.019' into merge-b4bdev-20250116
samsrabin Jan 16, 2025
c9e2ca8
Fix conflicts in lilac/lnd_comp_esmf.F90 with 019 version.
samsrabin Jan 16, 2025
031407c
lilac/lnd_comp_esmf.F90: Switch stop/restart alarm order.
samsrabin Jan 16, 2025
6194751
Update MOSART and RTM to fix nag build error.
samsrabin Jan 16, 2025
b427fc9
Remove two expected failures due to issue ESCOMP/CTSM#2914.
samsrabin Jan 17, 2025
bf65082
Remove an expected failure due to ESCOMP/CTSM#2905.
samsrabin Jan 17, 2025
bc03504
Remove failures expected due to ESCOMP/CTSM#2454.
samsrabin Jan 17, 2025
ff2ac19
Update ChangeLog/ChangeSum.
samsrabin Jan 17, 2025
a4aa4ec
Merge pull request #2938 from samsrabin/merge-b4bdev-20250116
samsrabin Jan 17, 2025
b8e71e5
Merge tag 'ctsm5.3.020' into merge-master-20250116
samsrabin Jan 17, 2025
ec0986b
Merge pull request #2939 from samsrabin/merge-master-20250116
samsrabin Jan 17, 2025
a1765aa
Merge branch 'ESCOMP:master' into fix_urbantvmapalgo_broadcast
briandobbins Jan 22, 2025
3360409
Adds the 'redist' method for the urban time-varying data, which is cu…
briandobbins Jan 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -27,6 +27,7 @@ d866510188d26d51bcd6d37239283db690af7e82
e096358c832ab292ddfd22dd5878826c7c788968
475831f0fb0e31e97f630eac4e078c886558b61c
fd5f177131d63d39e79a13918390bdfb642d781e
a51816e0de380300b69db9fc3e2c7fa83b267b64
# Ran SystemTests and python/ctsm through black python formatter
5364ad66eaceb55dde2d3d598fe4ce37ac83a93c
8056ae649c1b37f5e10aaaac79005d6e3a8b2380
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -44,15 +44,15 @@ fxDONOTUSEurl = https://github.com/ESCOMP/CISM-wrapper
[submodule "rtm"]
path = components/rtm
url = https://github.com/ESCOMP/RTM
fxtag = rtm1_0_80
fxtag = rtm1_0_84
fxrequired = ToplevelRequired
# Standard Fork to compare to with "git fleximod test" to ensure personal forks aren't committed
fxDONOTUSEurl = https://github.com/ESCOMP/RTM

[submodule "mosart"]
path = components/mosart
url = https://github.com/ESCOMP/MOSART
fxtag = mosart1.1.02
fxtag = mosart1.1.06
fxrequired = ToplevelRequired
# Standard Fork to compare to with "git fleximod test" to ensure personal forks aren't committed
fxDONOTUSEurl = https://github.com/ESCOMP/MOSART
108 changes: 108 additions & 0 deletions doc/ChangeLog
Original file line number Diff line number Diff line change
@@ -1,4 +1,112 @@
===============================================================
Tag name: ctsm5.3.018
Originator(s): slevis (Samuel Levis,UCAR/TSS,303-665-1310)
Date: Fri 10 Jan 2025 05:37:08 PM MST
One-line Summary: Change history time to be the middle of the time bounds

Purpose and description of changes
----------------------------------
Making the change to be consistent with CAM and to make history output more intuitive.

Significant changes to scientifically-supported configurations
--------------------------------------------------------------

Does this tag change answers significantly for any of the following physics configurations?
(Details of any changes will be given in the "Answer changes" section below.)

[Put an [X] in the box for any configuration with significant answer changes.]

[ ] clm6_0

[ ] clm5_1

[ ] clm5_0

[ ] ctsm5_0-nwp

[ ] clm4_5


Bugs fixed
----------
List of CTSM issues fixed (include CTSM Issue # and description) [one per line]:
Partly addresses issue #1059

Notes of particular relevance for users
---------------------------------------
Caveats for users (e.g., need to interpolate initial conditions):
The history time variable now equals the middle of the time bounds.
Instantaneous history tapes now do not include time bounds.
Mixed history tapes do not change the treatment of instantaneous fields or move them to separate tapes, yet.

Notes of particular relevance for developers:
---------------------------------------------
Caveats for developers (e.g., code that is duplicated that requires double maintenance):
Same changes are needed separately in clm, in mosart, and in rtm.

Changes to tests or testing:
This tag introduces changes to the mosart/rtm testlists.

FAIL RXCROPMATURITYSKIPGEN_Ld1097.f10_f10_mg37.IHistClm60BgcCrop.derecho_intel.clm-cropMonthOutput RUN
I did not label this failure EXPECTED because the fix comes in later in this series of "history" tags, in particular ctsm5.3.020.

I resolved the izumi nag tests that failed to build (due to a bug in rtm and mosart) by introducing the bug-fix manually, as explained here:
https://github.com/ESCOMP/CTSM/pull/2084#issuecomment-2584164690
In the next tag we expect to update to the rtm/mosart tags that include the fix.

Testing summary:
----------------

[PASS means all tests PASS; OK means tests PASS other than expected fails.]

build-namelist tests

derecho - PASS

python testing (if python code has changed; see instructions in python/README.md; document testing done):

derecho - PASS

regular tests (aux_clm: https://github.com/ESCOMP/CTSM/wiki/System-Testing-Guide#pre-merge-system-testing):

derecho ----- OK
izumi ------- OK

mosart
derecho ----- OK
izumi ------- OK

rtm
derecho ----- OK

Answer changes
--------------

Changes answers relative to baseline: Only time variable, plus read caveat

Summarize any changes to answers, i.e.,
- what code configurations: all
- what platforms/compilers: all
- nature of change: only the time variable

Caveat: We see diffs in mosart and cpl output that will be eliminated later in this series of "history" tags, in particular ctsm5.3.020. They are discussed here:
https://github.com/ESCOMP/CTSM/pull/2838#issuecomment-2477608383
https://github.com/ESCOMP/MOSART/issues/103#issuecomment-2479679014

Other details
-------------
List any git submodules updated (cime, rtm, mosart, cism, fates, etc.):
rtm, mosart

Pull Requests that document the changes (include PR ids):
https://github.com/ESCOMP/ctsm/pull/2838
https://github.com/ESCOMP/MOSART/pull/70
https://github.com/ESCOMP/RTM/issues/54
https://github.com/ESCOMP/MOSART/pull/106
https://github.com/ESCOMP/RTM/pull/39

===============================================================
===============================================================
Tag name: ctsm5.3.017
Originator(s): slevis (Samuel Levis,UCAR/TSS,303-665-1310)
Date: Thu 09 Jan 2025 11:56:43 AM MST
1 change: 1 addition & 0 deletions doc/ChangeSum
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
Tag Who Date Summary
============================================================================================================================
ctsm5.3.018 slevis 01/10/2025 Change history time to be the middle of the time bounds
ctsm5.3.017 slevis 01/09/2025 Merge tmp-241219 branch to master
tmp-241219.n03.ctsm5.3.016 01/09/2025 Bug fix for izumi nag tests to pass (slevis)
tmp-241219.n02.ctsm5.3.016 01/08/2025 FATES hydro test update (glemieux)
34 changes: 30 additions & 4 deletions python/ctsm/crop_calendars/cropcal_module.py
Original file line number Diff line number Diff line change
@@ -443,10 +443,7 @@ def import_output(
)

# Convert time axis to integer year, saving original as 'cftime'
this_ds_gs = this_ds_gs.assign_coords(
{"cftime": this_ds["time_bounds"].isel({"hist_interval": 0})}
)
this_ds_gs = this_ds_gs.assign_coords({"time": [t.year for t in this_ds_gs["cftime"].values]})
this_ds_gs = convert_time_to_int_year(filename, this_ds, this_ds_gs)

# Get number of harvests
this_ds_gs["NHARVESTS"] = (this_ds_gs["GDDHARV_PERHARV"] > 0).sum(dim="mxharvests")
@@ -458,6 +455,35 @@ def import_output(
return this_ds_gs, any_bad


def convert_time_to_int_year(filename, this_ds, this_ds_gs):
"""
Convert time axis to integer year, saving original as 'cftime'
"""
if "time_bounds" in this_ds:
# Always true before PR #2838, when even files with all instantaneous variables got
# time_bounds saved. After that PR (and before the segregation of instantaneous and other
# variables onto separate files), files with an instantaneous variable first in their list
# do not get time_bounds saved.
this_ds_gs = this_ds_gs.assign_coords(
{"cftime": this_ds["time_bounds"].isel({"hist_interval": 0})}
)
this_ds_gs = this_ds_gs.assign_coords(
{"time": [t.year for t in this_ds_gs["cftime"].values]}
)
elif this_ds["time"].attrs["long_name"] == "time at end of time step":
# This is an "instantaneous file."
this_ds_gs = this_ds_gs.assign_coords({"cftime": this_ds["time"]})
this_ds_gs = this_ds_gs.assign_coords(
{"time": [t.year - 1 for t in this_ds_gs["cftime"].values]}
)
else:
raise RuntimeError(
f"{filename} is neither an instantaneous nor a combined/non-instantaneous file."
)

return this_ds_gs


def handle_zombie_crops(this_ds):
"""
When doing transient runs, it's somehow possible for crops in newly-active patches to be
61 changes: 47 additions & 14 deletions src/main/histFileMod.F90
Original file line number Diff line number Diff line change
@@ -1172,6 +1172,7 @@ subroutine htape_addfld (t, f, avgflag)
integer :: beg1d,end1d ! beginning and ending indices for this field (assume already set)
integer :: num1d_out ! history output 1d size
type(bounds_type) :: bounds
character(len=avgflag_strlen) :: avgflag_temp ! local copy of hist_avgflag_pertape(t)
character(len=*),parameter :: subname = 'htape_addfld'
!-----------------------------------------------------------------------

@@ -1302,6 +1303,19 @@ subroutine htape_addfld (t, f, avgflag)
tape(t)%hlist(n)%avgflag = avgflag
end if

! Override this tape's avgflag if nhtfrq == 1
if (tape(t)%nhtfrq == 1) then ! output is instantaneous
hist_avgflag_pertape(t) = 'I'
end if
! Override this field's avgflag if the namelist or the previous line
! has set this tape to
! - instantaneous (I) or
! - local time (L)
avgflag_temp = hist_avgflag_pertape(t)
if (avgflag_temp == 'I' .or. avgflag_temp(1:1) == 'L') then
tape(t)%hlist(n)%avgflag = avgflag_temp
end if

end subroutine htape_addfld

!-----------------------------------------------------------------------
@@ -3098,6 +3112,7 @@ subroutine htape_timeconst(t, mode)
integer :: mcdate ! current date
integer :: yr,mon,day,nbsec ! year,month,day,seconds components of a date
integer :: hours,minutes,secs ! hours,minutes,seconds of hh:mm:ss
character(len= 12) :: step_or_bounds ! string used in long_name of several time variables
character(len= 10) :: basedate ! base date (yyyymmdd)
character(len= 8) :: basesec ! base seconds
character(len= 8) :: cdate ! system date
@@ -3357,20 +3372,30 @@ subroutine htape_timeconst(t, mode)

dim1id(1) = time_dimid
str = 'days since ' // basedate // " " // basesec
call ncd_defvar(nfid(t), 'time', tape(t)%ncprec, 1, dim1id, varid, &
long_name='time',units=str)
if (hist_avgflag_pertape(t) /= 'I') then ! NOT instantaneous fields tape
step_or_bounds = 'time_bounds'
long_name = 'time at exact middle of ' // step_or_bounds
call ncd_defvar(nfid(t), 'time', tape(t)%ncprec, 1, dim1id, varid, &
long_name=long_name, units=str)
call ncd_putatt(nfid(t), varid, 'bounds', 'time_bounds')
else ! instantaneous fields tape
step_or_bounds = 'time step'
long_name = 'time at end of ' // step_or_bounds
call ncd_defvar(nfid(t), 'time', tape(t)%ncprec, 1, dim1id, varid, &
long_name=long_name, units=str)
end if
cal = get_calendar()
if ( trim(cal) == NO_LEAP_C )then
caldesc = "noleap"
else if ( trim(cal) == GREGORIAN_C )then
caldesc = "gregorian"
end if
call ncd_putatt(nfid(t), varid, 'calendar', caldesc)
call ncd_putatt(nfid(t), varid, 'bounds', 'time_bounds')

dim1id(1) = time_dimid
long_name = 'current date (YYYYMMDD) at end of ' // step_or_bounds
call ncd_defvar(nfid(t) , 'mcdate', ncd_int, 1, dim1id , varid, &
long_name = 'current date (YYYYMMDD)')
long_name = long_name)
!
! add global attribute time_period_freq
!
@@ -3397,18 +3422,23 @@ subroutine htape_timeconst(t, mode)
call ncd_putatt(nfid(t), ncd_global, 'time_period_freq', &
trim(time_period_freq))

long_name = 'current seconds of current date at end of ' // step_or_bounds
call ncd_defvar(nfid(t) , 'mcsec' , ncd_int, 1, dim1id , varid, &
long_name = 'current seconds of current date', units='s')
long_name = long_name, units='s')
long_name = 'current day (from base day) at end of ' // step_or_bounds
call ncd_defvar(nfid(t) , 'mdcur' , ncd_int, 1, dim1id , varid, &
long_name = 'current day (from base day)')
long_name = long_name)
long_name = 'current seconds of current day at end of ' // step_or_bounds
call ncd_defvar(nfid(t) , 'mscur' , ncd_int, 1, dim1id , varid, &
long_name = 'current seconds of current day')
long_name = long_name)
call ncd_defvar(nfid(t) , 'nstep' , ncd_int, 1, dim1id , varid, &
long_name = 'time step')

dim2id(1) = hist_interval_dimid; dim2id(2) = time_dimid
call ncd_defvar(nfid(t), 'time_bounds', ncd_double, 2, dim2id, varid, &
long_name = 'history time interval endpoints')
if (hist_avgflag_pertape(t) /= 'I') then ! NOT instantaneous fields tape
call ncd_defvar(nfid(t), 'time_bounds', ncd_double, 2, dim2id, varid, &
long_name = 'history time interval endpoints')
end if

dim2id(1) = strlen_dimid; dim2id(2) = time_dimid
call ncd_defvar(nfid(t), 'date_written', ncd_char, 2, dim2id, varid)
@@ -3436,13 +3466,16 @@ subroutine htape_timeconst(t, mode)
call ncd_io('mscur' , mscur , 'write', nfid(t), nt=tape(t)%ntimes)
call ncd_io('nstep' , nstep , 'write', nfid(t), nt=tape(t)%ntimes)

time = mdcur + mscur/secspday
timedata(1) = tape(t)%begtime ! beginning time
timedata(2) = mdcur + mscur/secspday ! end time
if (hist_avgflag_pertape(t) /= 'I') then ! NOT instantaneous fields tape
time = (timedata(1) + timedata(2)) * 0.5_r8
call ncd_io('time_bounds', timedata, 'write', nfid(t), nt=tape(t)%ntimes)
else
time = timedata(2)
end if
call ncd_io('time' , time , 'write', nfid(t), nt=tape(t)%ntimes)

timedata(1) = tape(t)%begtime
timedata(2) = time
call ncd_io('time_bounds', timedata, 'write', nfid(t), nt=tape(t)%ntimes)

call getdatetime (cdate, ctime)
call ncd_io('date_written', cdate, 'write', nfid(t), nt=tape(t)%ntimes)