Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds MAM4xx Surface emissions process interface for EAMxx #2944

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
0ae7719
Adds surface and online emission interface and a test
singhbalwinder Feb 21, 2024
0223b78
Adds some inputs and preprocess struct, test works now
singhbalwinder Feb 22, 2024
30f0494
Adds some notes for the integration
singhbalwinder Feb 22, 2024
2bdad85
A working test in new single-process test folder
singhbalwinder Jul 8, 2024
9948fa8
Adds horiz wind vector and some cleanup; test is working
singhbalwinder Jul 8, 2024
c9f8c52
get scream in correct place for emissions
mjs271 Apr 15, 2024
407c5f3
compiling and running test
mjs271 Jul 8, 2024
b30d5fd
reading files but scorpio issues with freeing file
mjs271 Jul 12, 2024
79b88b7
surface and online emissions data files successfull read in
mjs271 Jul 18, 2024
f5da245
Horizontal remapper create_horiz_remapper works with so2 srf file
singhbalwinder Jul 24, 2024
261ea39
Adds srfmiss input
singhbalwinder Jul 24, 2024
cb28902
Adds a call to create_srfEmiss_data_reader
singhbalwinder Jul 24, 2024
7a37ccf
update_srfEmiss_data_from_file call in initialize_impl works
singhbalwinder Jul 25, 2024
7b2c9b2
enables update_srfEmiss_timestate in run_impl
singhbalwinder Jul 25, 2024
0827262
Implements the main routine to do the interpolation in run_impl
singhbalwinder Jul 25, 2024
d01af51
Adds so2_fields array as an arg, removes iop code
singhbalwinder Jul 25, 2024
f7c7e77
Adds a template arg for string arrays carrying field names
singhbalwinder Jul 25, 2024
b9e7e3d
Adds code to remove explict mention of specie names
singhbalwinder Jul 25, 2024
0de559d
Removed emissions sector names from all files
singhbalwinder Jul 26, 2024
3d8c98c
Removes extra codes and kept code only for reading file
singhbalwinder Jul 26, 2024
2d616f7
Removes hardwired num sectors
singhbalwinder Jul 26, 2024
343bfc3
Creates a function to init srf emiss data structures
singhbalwinder Jul 26, 2024
4ae7b5a
Sums up all emission sectors
singhbalwinder Jul 26, 2024
9769a95
Adds bc_a4 and dms emmisions
singhbalwinder Jul 26, 2024
14e944f
Adds all the emission files
singhbalwinder Jul 26, 2024
b073225
Computes emissions flux and adds use of molecular weights for unit co…
singhbalwinder Jul 31, 2024
89ec5e5
Removes num_sectors from the arg list
singhbalwinder Jul 31, 2024
5db0b8a
Partial implementation of a vector for surface emissions
singhbalwinder Jul 31, 2024
fecc309
The vector for surface emissions is working in a for loop
singhbalwinder Aug 1, 2024
4f81656
Fixes compilation issues in previous commit
singhbalwinder Aug 1, 2024
f3aad49
Updates constituent flux array with the righ units
singhbalwinder Aug 1, 2024
5b35e31
Adds constituent flixes interface and a test
singhbalwinder Aug 1, 2024
b8ff360
Adds code to create progs and atm datastructures
singhbalwinder Aug 2, 2024
552a785
A working end-to-end code
singhbalwinder Aug 2, 2024
a3c0ab9
Verified change in DMS conc before and after update, moved aci rpdel …
singhbalwinder Aug 2, 2024
3496c7d
Cleanup of some unused hpp file includes and some variables
singhbalwinder Aug 2, 2024
4048ae8
Removes unused variables
singhbalwinder Aug 2, 2024
0b9b81d
Cleanup - removed unused codes and added kokkos reduction loop
singhbalwinder Aug 2, 2024
0c258d4
Adds output fileds to yaml and also change the location of emissions …
singhbalwinder Aug 2, 2024
c29bab5
Some GPU fixes but needs some cleanup
singhbalwinder Aug 3, 2024
e4b0aa2
Reworked logic to add fluxes only at the surface level-needs cleanup
singhbalwinder Aug 4, 2024
4c5f58c
Fixes pm-gpu build and run-needs cleanup
singhbalwinder Aug 4, 2024
209ba3d
Moved Kokkos - loop outside of interface and verified that gas and ae…
singhbalwinder Aug 6, 2024
d4c9e2f
Fixes some GPU errors and warnings
singhbalwinder Aug 9, 2024
6f1e5d7
Fix some compile warnings that are really errors.
overfelt Aug 10, 2024
9c2fb65
Fixes a warning for a size type for a string vector
singhbalwinder Aug 13, 2024
c45a86a
Clang format
singhbalwinder Aug 13, 2024
245571a
Store size in local in to avoid declaring long size_type datatype
singhbalwinder Aug 13, 2024
029ba9c
clang format
singhbalwinder Aug 13, 2024
a816871
Revert "Merge branch 'master' into singhbalwinder/branch_mjs/mam4xx/e…
singhbalwinder Aug 13, 2024
a7e7191
Rebases and update mam4 submodule to point to its latest master
singhbalwinder Aug 13, 2024
f1b2ab6
Adds a multi-process test with srf emiss and constituent fluxes
singhbalwinder Aug 14, 2024
2bf2661
Adds testmods and files for ne4pg2 CIME simulation
singhbalwinder Aug 14, 2024
5879c57
Revertes codes that were changed after the rebase
singhbalwinder Aug 15, 2024
1d796b4
Changes emission files that use conservative mapping
singhbalwinder Aug 15, 2024
7316181
Fixes multi process test a typo in aci interface
singhbalwinder Aug 16, 2024
c0a608e
Adds ne30pg2 emissions files and mapping files for fine resolutions
singhbalwinder Aug 16, 2024
4fa577e
Removes debug print statements
singhbalwinder Aug 17, 2024
87ea43c
Edits and adds some comments to provide clarity and TODOs
singhbalwinder Aug 17, 2024
050d149
Fixes namelist to enable fine grid simulations
singhbalwinder Aug 28, 2024
d8d89e6
Update shell_commands to remove spa
singhbalwinder Aug 29, 2024
4b57352
Adds units for mass and number ofr constituent fluxes
singhbalwinder Aug 29, 2024
215d225
Fixes path for files and change c-style strings to std::strings
singhbalwinder Sep 1, 2024
b1fbda1
Replaces preprocess interface code
singhbalwinder Sep 1, 2024
c6d0711
Removes preprocess function from hpp and a calls from cpp interface f…
singhbalwinder Sep 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 36 additions & 4 deletions components/eamxx/cime_config/namelist_defaults_scream.xml
Original file line number Diff line number Diff line change
Expand Up @@ -242,8 +242,7 @@ be lost if SCREAM_HACK_XML is not enabled.
</mam4_aci>

<!-- MAM4xx-Dry Deposition -->
<mam4_drydep inherit="atm_proc_base">
</mam4_drydep>
<mam4_drydep inherit="atm_proc_base" />

<!-- CLD fraction -->
<cldFraction inherit="atm_proc_base"/>
Expand All @@ -266,8 +265,41 @@ be lost if SCREAM_HACK_XML is not enabled.
</mam4_optics>

<!-- MAM4xx-Wetscav -->
<mam4_wetscav inherit="atm_proc_base">
</mam4_wetscav>
<mam4_wetscav inherit="atm_proc_base" />

<!-- MAM4xx-Surface-Emissions -->
<mam4_srf_online_emiss inherit="atm_proc_base">

<srf_emis_specifier_for_DMS hgrid="ne4np4.pg2" type="file" doc="File containing surface emissions data for DMS">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne4pg2/surface/DMSflux.2010.ne4pg2_conserv.POPmonthlyClimFromACES4BGC_c20240814.nc</srf_emis_specifier_for_DMS>
<srf_emis_specifier_for_SO2 hgrid="ne4np4.pg2" type="file" doc="File containing surface emissions data for SO2">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne4pg2/surface/cmip6_mam4_so2_surf_ne4pg2_2010_clim_c20240815.nc</srf_emis_specifier_for_SO2>
kaizhangpnl marked this conversation as resolved.
Show resolved Hide resolved
<srf_emis_specifier_for_bc_a4 hgrid="ne4np4.pg2" type="file" doc="File containing surface emissions data for bc_a4">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne4pg2/surface/cmip6_mam4_bc_a4_surf_ne4pg2_2010_clim_c20240815.nc</srf_emis_specifier_for_bc_a4>
<srf_emis_specifier_for_num_a1 hgrid="ne4np4.pg2" type="file" doc="File containing surface emissions data for num_a1">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne4pg2/surface/cmip6_mam4_num_a1_surf_ne4pg2_2010_clim_c20240815.nc</srf_emis_specifier_for_num_a1>
<srf_emis_specifier_for_num_a2 hgrid="ne4np4.pg2" type="file" doc="File containing surface emissions data for num_a2">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne4pg2/surface/cmip6_mam4_num_a2_surf_ne4pg2_2010_clim_c20240815.nc</srf_emis_specifier_for_num_a2>
<srf_emis_specifier_for_num_a4 hgrid="ne4np4.pg2" type="file" doc="File containing surface emissions data for num_a4">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne4pg2/surface/cmip6_mam4_num_a4_surf_ne4pg2_2010_clim_c20240815.nc</srf_emis_specifier_for_num_a4>
<srf_emis_specifier_for_pom_a4 hgrid="ne4np4.pg2" type="file" doc="File containing surface emissions data for pom_a4">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne4pg2/surface/cmip6_mam4_pom_a4_surf_ne4pg2_2010_clim_c20240815.nc</srf_emis_specifier_for_pom_a4>
<srf_emis_specifier_for_so4_a1 hgrid="ne4np4.pg2" type="file" doc="File containing surface emissions data for so4_a1">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne4pg2/surface/cmip6_mam4_so4_a1_surf_ne4pg2_2010_clim_c20240815.nc</srf_emis_specifier_for_so4_a1>
<srf_emis_specifier_for_so4_a2 hgrid="ne4np4.pg2" type="file" doc="File containing surface emissions data for so4_a2">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne4pg2/surface/cmip6_mam4_so4_a2_surf_ne4pg2_2010_clim_c20240815.nc</srf_emis_specifier_for_so4_a2>
<!-- For all other grids -->
<srf_emis_specifier_for_DMS type="file" doc="File containing surface emissions data for DMS">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne30pg2/surface/DMSflux.2010.ne30pg2_conserv.POPmonthlyClimFromACES4BGC_c20240816.nc</srf_emis_specifier_for_DMS>
<srf_emis_specifier_for_SO2 type="file" doc="File containing surface emissions data for SO2">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne30pg2/surface/cmip6_mam4_so2_surf_ne30pg2_2010_clim_c20240816.nc</srf_emis_specifier_for_SO2>
<srf_emis_specifier_for_bc_a4 type="file" doc="File containing surface emissions data for bc_a4">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne30pg2/surface/cmip6_mam4_bc_a4_surf_ne30pg2_2010_clim_c20240816.nc</srf_emis_specifier_for_bc_a4>
<srf_emis_specifier_for_num_a1 type="file" doc="File containing surface emissions data for num_a1">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne30pg2/surface/cmip6_mam4_num_a1_surf_ne30pg2_2010_clim_c20240816.nc</srf_emis_specifier_for_num_a1>
<srf_emis_specifier_for_num_a2 type="file" doc="File containing surface emissions data for num_a2">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne30pg2/surface/cmip6_mam4_num_a2_surf_ne30pg2_2010_clim_c20240816.nc</srf_emis_specifier_for_num_a2>
<srf_emis_specifier_for_num_a4 type="file" doc="File containing surface emissions data for num_a4">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne30pg2/surface/cmip6_mam4_num_a4_surf_ne30pg2_2010_clim_c20240816.nc</srf_emis_specifier_for_num_a4>
<srf_emis_specifier_for_pom_a4 type="file" doc="File containing surface emissions data for pom_a4">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne30pg2/surface/cmip6_mam4_pom_a4_surf_ne30pg2_2010_clim_c20240816.nc</srf_emis_specifier_for_pom_a4>
<srf_emis_specifier_for_so4_a1 type="file" doc="File containing surface emissions data for so4_a1">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne30pg2/surface/cmip6_mam4_so4_a1_surf_ne30pg2_2010_clim_c20240816.nc</srf_emis_specifier_for_so4_a1>
<srf_emis_specifier_for_so4_a2 type="file" doc="File containing surface emissions data for so4_a2">${DIN_LOC_ROOT}/atm/scream/mam4xx/emissions/ne30pg2/surface/cmip6_mam4_so4_a2_surf_ne30pg2_2010_clim_c20240816.nc</srf_emis_specifier_for_so4_a2>

<!-- Mapping Files for finer resolutions -->
<srf_remap_file type="file" doc="File containing mapping data from the grid of emission files to the model grid. Unused if the grid is the same."/>
<srf_remap_file hgrid="ne120np4.pg2">${DIN_LOC_ROOT}/atm/scream/maps/map_ne30pg2_to_ne120pg2_20231201.nc</srf_remap_file>
<srf_remap_file hgrid="ne256np4.pg2">${DIN_LOC_ROOT}/atm/scream/maps/map_ne30pg2_to_ne256pg2_20231201.nc</srf_remap_file>
<srf_remap_file hgrid="ne512np4.pg2">${DIN_LOC_ROOT}/atm/scream/maps/map_ne30pg2_to_ne512pg2_20231201.nc</srf_remap_file>
<srf_remap_file hgrid="ne1024np4.pg2">${DIN_LOC_ROOT}/atm/scream/maps/map_ne30pg2_to_ne1024pg2_20231201.nc</srf_remap_file>
</mam4_srf_online_emiss>

<!-- MAM4xx-constituent fluxes -->
<mam4_constituent_fluxes inherit="atm_proc_base" />

<!-- nudging -->
<nudging inherit="atm_proc_base">
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@

#!/bin/sh
#------------------------------------------------------
# MAM4xx adds additionaltracers to the simulation
# Increase number of tracers for MAM4xx simulations
#------------------------------------------------------

$CIMEROOT/../components/eamxx/cime_config/testdefs/testmods_dirs/scream/mam4xx/update_eamxx_num_tracers.sh -b

#------------------------------------------------------
#Update IC file and add the processes
#------------------------------------------------------
$CIMEROOT/../components/eamxx/scripts/atmchange initial_conditions::Filename='$DIN_LOC_ROOT/atm/scream/init/screami_mam4xx_ne4np4L72_c20240208.nc' -b
$CIMEROOT/../components/eamxx/scripts/atmchange physics::atm_procs_list="mam4_constituent_fluxes,mac_aero_mic,rrtmgp,mam4_srf_online_emiss" -b



4 changes: 3 additions & 1 deletion components/eamxx/src/physics/mam/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,9 @@ add_library(mam
eamxx_mam_optics_process_interface.cpp
eamxx_mam_dry_deposition_process_interface.cpp
eamxx_mam_aci_process_interface.cpp
eamxx_mam_wetscav_process_interface.cpp)
eamxx_mam_wetscav_process_interface.cpp
eamxx_mam_srf_and_online_emissions_process_interface.cpp
eamxx_mam_constituent_fluxes_interface.cpp)
target_compile_definitions(mam PUBLIC EAMXX_HAS_MAM)
add_dependencies(mam mam4xx)
target_include_directories(mam PUBLIC
Expand Down
19 changes: 1 addition & 18 deletions components/eamxx/src/physics/mam/eamxx_mam_aci_functions.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -198,23 +198,6 @@ void store_liquid_cloud_fraction(
});
}

void compute_recipical_pseudo_density(haero::ThreadTeamPolicy team_policy,
MAMAci::const_view_2d pdel,
const int nlev,
// output
MAMAci::view_2d rpdel) {
Kokkos::parallel_for(
team_policy, KOKKOS_LAMBDA(const haero::ThreadTeam &team) {
const int icol = team.league_rank();
Kokkos::parallel_for(
Kokkos::TeamVectorRange(team, 0, nlev), [&](int kk) {
EKAT_KERNEL_ASSERT_MSG(0 < pdel(icol, kk),
"Error: pdel should be > 0.\n");
rpdel(icol, kk) = 1 / pdel(icol, kk);
});
});
}

void call_function_dropmixnuc(
haero::ThreadTeamPolicy team_policy, const Real dt,
mam_coupling::DryAtmosphere &dry_atmosphere, const MAMAci::view_2d rpdel,
Expand Down Expand Up @@ -397,7 +380,7 @@ void call_function_dropmixnuc(
progs_at_col, haero_atm, state_q_at_lev_col, klev);

// get the start index for aerosols species in the state_q array
int istart = mam4::aero_model::pcnst - mam4::ndrop::ncnst_tot;
int istart = mam4::utils::aero_start_ind();

// create colum views of state_q
for(int icnst = istart; icnst < mam4::aero_model::pcnst;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ void MAMAci::set_grids(
frz_unit, grid_name);

// heterogeneous freezing by deposition nucleation [cm^-3 s^-1]
add_field<Computed>("hetfrz_depostion_nucleation_tend", scalar3d_layout_mid,
add_field<Computed>("hetfrz_deposition_nucleation_tend", scalar3d_layout_mid,
frz_unit, grid_name);
} // function set_grids ends

Expand Down Expand Up @@ -390,8 +390,8 @@ void MAMAci::initialize_impl(const RunType run_type) {
get_field_out("hetfrz_immersion_nucleation_tend").get_view<Real **>();
hetfrz_contact_nucleation_tend_ =
get_field_out("hetfrz_contact_nucleation_tend").get_view<Real **>();
hetfrz_depostion_nucleation_tend_ =
get_field_out("hetfrz_depostion_nucleation_tend").get_view<Real **>();
hetfrz_deposition_nucleation_tend_ =
get_field_out("hetfrz_deposition_nucleation_tend").get_view<Real **>();

//---------------------------------------------------------------------------------
// Allocate memory for the class members
Expand Down Expand Up @@ -597,7 +597,7 @@ void MAMAci::run_impl(const double dt) {
// output
cloud_frac_, cloud_frac_prev_);

compute_recipical_pseudo_density(team_policy, dry_atm_.p_del, nlev_,
mam_coupling::compute_recipical_pseudo_density(team_policy, dry_atm_.p_del, nlev_,
// output
rpdel_);

Expand Down Expand Up @@ -642,7 +642,7 @@ void MAMAci::run_impl(const double dt) {
team_policy, hetfrz_, dry_atm_, dry_aero_, factnum_, dt, nlev_,
// ## output to be used by the other processes ##
hetfrz_immersion_nucleation_tend_, hetfrz_contact_nucleation_tend_,
hetfrz_depostion_nucleation_tend_,
hetfrz_deposition_nucleation_tend_,
// work arrays
diagnostic_scratch_);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ class MAMAci final : public scream::AtmosphereProcess {
// added correctly to the cloud-micorphysics scheme.
view_2d hetfrz_immersion_nucleation_tend_;
view_2d hetfrz_contact_nucleation_tend_;
view_2d hetfrz_depostion_nucleation_tend_;
view_2d hetfrz_deposition_nucleation_tend_;

view_2d diagnostic_scratch_[hetro_scratch_];

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#ifndef EAMXX_MAM_CONSTITUTE_FLUXES_FUNCTIONS_HPP
#define EAMXX_MAM_CONSTITUTE_FLUXES_FUNCTIONS_HPP

#include <mam4xx/mam4.hpp>
#include <physics/mam/eamxx_mam_constituent_fluxes_interface.hpp>

namespace scream {

namespace {

void update_gas_aerosols_using_constituents(
const int ncol, const int nlev, const double dt,
const mam_coupling::DryAtmosphere &dry_atm,
const MAMConstituentFluxes::const_view_2d &constituent_fluxes,
// output
const mam_coupling::AerosolState &wet_aero) {
using C = physics::Constants<Real>;
static constexpr auto gravit = C::gravit; // Gravity [m/s2]
static constexpr int pcnst = mam4::aero_model::pcnst;

// Declare local variables
const int surface_lev = nlev - 1;

// get the start index for gas species in the state_q array
int istart = mam4::utils::gasses_start_ind();

// number of constituents to update (currently updating only MAM4xx
// constituents)
const int nconstituents = pcnst - istart;

// Create a policy to loop over columns annd number of constituents
// to update
// FIXME: TODO:We don't need a team for "nconstituents", so we can make the
// kookos_for simple by using just ncols
const auto policy = ekat::ExeSpaceUtils<MAMConstituentFluxes::KT::ExeSpace>::
get_default_team_policy(ncol, nconstituents);

// Loop through all columns to update tracer mixing rations
Kokkos::parallel_for(
policy, KOKKOS_LAMBDA(const haero::ThreadTeam &team) {
const int icol = team.league_rank();

//----------------------------------------------------------------------
// To form EAM like state%q array, we need prognostics (gas and aerosol
// mmrs) atmosphere (qv, qc, nc, ni, etc.)
//----------------------------------------------------------------------

// Get prognostics
mam4::Prognostics progs_at_col =
mam_coupling::aerosols_for_column(wet_aero, // output
icol); // input
// Get atmospheric quantities
const haero::Atmosphere haero_atm =
atmosphere_for_column(dry_atm, // output
icol); // input

// Form state%q like array at surface level
Real state_q_at_surf_lev[pcnst] = {};
mam4::utils::extract_stateq_from_prognostics(
progs_at_col, haero_atm, // input
state_q_at_surf_lev, // output
surface_lev); // input

// Compute the units conversion factor (kg/m2/s to kg/kg)
EKAT_KERNEL_ASSERT_MSG(dry_atm.p_del(icol, surface_lev) != 0,
"Error! dry_atm.pdel must be non-zero!\n");
const Real rpdel = 1.0 / dry_atm.p_del(icol, surface_lev);
const Real unit_factor = dt * gravit * rpdel;

// Update state vector with constituent fluxes
for(int icnst = istart; icnst < pcnst; ++icnst) {
state_q_at_surf_lev[icnst] +=
constituent_fluxes(icol, icnst) * unit_factor;
}
mam4::utils::inject_stateq_to_prognostics(state_q_at_surf_lev, // input
progs_at_col, // output
surface_lev); // input
}); // icol loop
}

} // namespace
} // namespace scream

#endif // ifdef EAMXX_MAM_CONSTITUTE_FLUXES_FUNCTIONS_HPP
Loading