Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify downstream atmos product generation scripts #1822

Merged
merged 23 commits into from
Sep 15, 2023
Merged

Simplify downstream atmos product generation scripts #1822

Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8db08a1
updates to atmos prodgen scripts for clarity
aerorahul Aug 23, 2023
3198cf9
all the cleanup that needed to happen in fv3gfs_downstream_nems.sh is…
aerorahul Aug 24, 2023
ee32c84
use COM_ATMOS_GRIB_GRID_TMPL
aerorahul Aug 24, 2023
c1cc3c9
Merge branch 'develop' into feature/atmos_products
aerorahul Aug 24, 2023
df9a7f8
Merge branch 'develop' into feature/atmos_products
aerorahul Aug 28, 2023
b45a2d6
use arrays, and loop over grids. remove unnecessary ifs
aerorahul Aug 30, 2023
7e6202a
fv3gfs_dwn_nems.sh does not create the 1-degree product anymore
aerorahul Aug 30, 2023
a19cddb
Apply suggestions from code review
aerorahul Aug 31, 2023
0b02c76
Apply suggestions from code review
aerorahul Aug 31, 2023
1330451
replace while loops w/ for and shellcheck fix
aerorahul Aug 31, 2023
53403ae
Merge branch 'develop' into feature/atmos_products
aerorahul Sep 13, 2023
eda8e73
add wcoss2. remove dependency on pGBS to be exported
aerorahul Sep 13, 2023
10ec4da
fix typos
aerorahul Sep 13, 2023
d3aa1cd
disable shellcheck as it is incorrect
aerorahul Sep 13, 2023
01d0263
fix issues on wcoss2
aerorahul Sep 14, 2023
b14d4a7
ensure the products in grp a are being archived. there will be furth…
aerorahul Sep 14, 2023
200e169
on slurm, use mpmd.jobid.taskid.out to collect MPMD output
aerorahul Sep 14, 2023
5ed9140
Merge branch 'develop' into feature/atmos_products
aerorahul Sep 14, 2023
59c5202
use CFP on jet and s4 for products
aerorahul Sep 14, 2023
1fece3f
fix forecast hour without f and remove mp_ stuff from mpmd
aerorahul Sep 14, 2023
ddd0c62
for now hold back on 2a and 2b
aerorahul Sep 14, 2023
0ad0ad3
tmpfilea does not exist
aerorahul Sep 14, 2023
02cc00e
remove unused variable
aerorahul Sep 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion env/HERA.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ step=$1

export npe_node_max=40
export launcher="srun -l --export=ALL"
export mpmd_opt="--multi-prog --output=${step}.%J.%t.out"
export mpmd_opt="--multi-prog --output=mpmd.%j.%t.out"

# Configure MPI environment
#export I_MPI_ADJUST_ALLREDUCE=5
Expand Down Expand Up @@ -215,6 +215,7 @@ elif [[ "${step}" = "post" ]]; then
[[ ${NTHREADS_NP} -gt ${nth_max} ]] && export NTHREADS_NP=${nth_max}
export APRUN_NP="${launcher} -n ${npe_post}"

export USE_CFP="YES" # Use MPMD for downstream product generation on Hera
export NTHREADS_DWN=${nth_dwn:-1}
[[ ${NTHREADS_DWN} -gt ${nth_max} ]] && export NTHREADS_DWN=${nth_max}
export APRUN_DWN="${launcher} -n ${npe_dwn}"
Expand Down Expand Up @@ -302,3 +303,4 @@ elif [[ "${step}" = "fit2obs" ]]; then
export MPIRUN="${launcher} -n ${npe_fit2obs}"

fi

2 changes: 1 addition & 1 deletion env/JET.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ elif [[ "${PARTITION_BATCH}" = "kjet" ]]; then
export npe_node_max=40
fi
export launcher="srun -l --epilog=/apps/local/bin/report-mem --export=ALL"
export mpmd_opt="--multi-prog --output=${step}.%J.%t.out"
export mpmd_opt="--multi-prog --output=mpmd.%j.%t.out"
aerorahul marked this conversation as resolved.
Show resolved Hide resolved

# Configure MPI environment
export OMP_STACKSIZE=2048000
Expand Down
3 changes: 2 additions & 1 deletion env/ORION.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ step=$1

export npe_node_max=40
export launcher="srun -l --export=ALL"
export mpmd_opt="--multi-prog --output=${step}.%J.%t.out"
export mpmd_opt="--multi-prog --output=mpmd.%j.%t.out"

# Configure MPI environment
export MPI_BUFS_PER_PROC=2048
Expand Down Expand Up @@ -214,6 +214,7 @@ elif [[ "${step}" = "post" ]]; then
[[ ${NTHREADS_NP} -gt ${nth_max} ]] && export NTHREADS_NP=${nth_max}
export APRUN_NP="${launcher} -n ${npe_post}"

export USE_CFP="YES" # Use MPMD for downstream product generation on Orion
export NTHREADS_DWN=${nth_dwn:-1}
[[ ${NTHREADS_DWN} -gt ${nth_max} ]] && export NTHREADS_DWN=${nth_max}
export APRUN_DWN="${launcher} -n ${npe_dwn}"
Expand Down
2 changes: 1 addition & 1 deletion env/S4.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ elif [[ ${PARTITION_BATCH} = "ivy" ]]; then
export npe_node_max=20
fi
export launcher="srun -l --export=ALL"
export mpmd_opt="--multi-prog --output=${step}.%J.%t.out"
export mpmd_opt="--multi-prog --output=mpmd.%j.%t.out"
aerorahul marked this conversation as resolved.
Show resolved Hide resolved

# Configure MPI environment
export OMP_STACKSIZE=2048000
Expand Down
1 change: 1 addition & 0 deletions env/WCOSS2.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,7 @@ elif [[ "${step}" = "post" ]]; then
[[ ${NTHREADS_NP} -gt ${nth_max} ]] && export NTHREADS_NP=${nth_max}
export APRUN_NP="${launcher} -n ${npe_np:-${npe_post}} -ppn ${npe_node_post} --cpu-bind depth --depth ${NTHREADS_NP}"

export USE_CFP="YES" # Use MPMD for downstream product generation on WCOSS2
export NTHREADS_DWN=${nth_dwn:-1}
[[ ${NTHREADS_DWN} -gt ${nth_max} ]] && export NTHREADS_DWN=${nth_max}
export APRUN_DWN="${launcher} -np ${npe_dwn} ${mpmd_opt}"
Expand Down
4 changes: 2 additions & 2 deletions jobs/JGLOBAL_ATMOS_POST
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ export PARMpost=${PARMpost:-${HOMEgfs}/parm/post}
export INLINE_POST=${WRITE_DOPOST:-".false."}

# Construct COM variables from templates
YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_RESTART COM_ATMOS_ANALYSIS COM_ATMOS_HISTORY COM_ATMOS_MASTER
YMD=${PDY} HH=${cyc} generate_com -rx COM_ATMOS_RESTART COM_ATMOS_ANALYSIS COM_ATMOS_HISTORY COM_ATMOS_MASTER COM_ATMOS_GRIB
aerorahul marked this conversation as resolved.
Show resolved Hide resolved
if [[ ! -d ${COM_ATMOS_MASTER} ]]; then mkdir -m 775 -p "${COM_ATMOS_MASTER}"; fi

if [[ ${GOESF} == "YES" ]]; then
Expand All @@ -51,7 +51,7 @@ fi
for grid in '0p25' '0p50' '1p00'; do
prod_dir="COM_ATMOS_GRIB_${grid}"
GRID=${grid} YMD=${PDY} HH=${cyc} generate_com -rx "${prod_dir}:COM_ATMOS_GRIB_GRID_TMPL"
if [[ ! -d "${prod_dir}" ]]; then mkdir -m 775 -p "${!prod_dir}"; fi
if [[ ! -d "${!prod_dir}" ]]; then mkdir -m 775 -p "${!prod_dir}"; fi
done

if [ "${RUN}" = gfs ];then
Expand Down
Loading