Converting Cubed Sphere Data to Lat Lon outside NASA HPC ecosystem

Prerequisites

To convert GEOS-IT NetCDF Cubed-Sphere data to a Lat-Lon gridded NetCDF file outside of the NASA HPC ecosystems users will need to install a few prerequisites to obtain tools that can regrid the data. Users will need Git installed to clone/download the software. Although not required you can use wget as detailed in the instructions below.

Overview of Method

The procedure and tools described below uses the ESMF (Earth System Modeling Framework, https://earthsystemmodeling.org/) offline regrid weight generation application to generate interpolation weights between two grids using a variety of options such as bilinear interpolation and conservative schemes. For those curious, more information about the interpolation methods and this tool can be found here: https://earthsystemmodeling.org/docs/release/ESMF_8_1_1/ESMF_refdoc/node3.html#SECTION03020000000000000000 Using the weights generated by this tool, they can be applied using the ncremap utility in NCO package (NetCDF operators, http://nco.sourceforge.net/). More information about this function can be found here: http://nco.sourceforge.net/nco.html#ncremap-netCDF-Remapper. As described below we have created Python scripting to automate the use of these tools for the user so it is not necessary to read the links above. They are there for curious users who may want a description of the remapping methods used.

ESMF/NCO Installation (via Python)

First you must install a python distribution that contains ESMF (Earth System Modeling Framework, https://earthsystemmodeling.org/) and NCO (NetCDF operators, http://nco.sourceforge.net/). The easiest way to do this is to use the Mamba installer for python. The installer can be obtained and installed in user space (no admin privileges). If you have wget installed on your system you can use that to obtain the installer from the command line. If you do not have wget these installer scripts referenced below can be downloaded here: https://github.com/conda-forge/miniforge/releases/latest, just find the one for your operating system as it described below.

To install this on Linux, run the following commands in a terminal where /path/to/install is a file path you have write permission for (usually somewhere in your HOME directory):

wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
bash Mambaforge-Linux-x86_64.sh -b -p /path/to/install
/path/to/install/bin/mamba install 'nco=4.9.9' 'esmf=8.1.1=*mpi*' netcdf4 scipy

or if you want you need ESMpy

path/to/install/bin/mamba install 'nco=5.0.1' 'esmf=8.2.0=*mpi*' netcdf4 scipy ESMPy mpi4py

This can also be installed on Mac OS for both Intel and the new M1 based processors from a terminal window. For Intel based Macs use this wget command:

wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Darwin-x86_64.sh

or for M1 based Macs use this wget command:

wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Darwin-arm64.sh

Then just run the install as in the Linux instructions (obviously use the correct name for the installer you downloaded!).

Finally we recommend that Windows 10 users install Windows Subsystem for Linux (WSL), which will allow them to run a GNU/Linux environment and follow the Linux instructions. A description of how to install WSL can be found here https://docs.microsoft.com/en-us/windows/wsl/install-win10. If you are unable to install WSL, there is a Windows installer for Mamba Python that you could use but we have not tested this.

Once you have installed this Python distribution, you will need to make sure that the bin directory in the installation is in your path.

GMAO Regridding Tools

Next you will need to obtain some small Python scripts maintained by the GMAO-SI team. They can be obtained with the following Git command:

git clone https://github.com/GMAO-SI-Team/regridding-tools.git

This will create a directory named regridding-tools where you will find several Python scripts that automate the use of the ESMF and NCO tools available for regridding. They handle the following tasks for the user:

• creating the scrip representation of the Cubed-Sphere grid from the input file
• converting the input file to the scrip format
• creating a descriptor file for the lat-lon grid
• running ESMF_RegridWeightGen to generate the regridding weights based on the previous two files
• using the weights to run ncremap to do the regridding with the converted input file
• running ncks to clean up unnecessary variables in the output.
• cleaning up temporary files

Usage

Once you have installed Python and cloned the regridding tools, in the regridding-tools directory you will find a convert_tool.py which is the Python script you will use from the command line in a terminal to perform the regridding. It allows the user to give an input file and a target lat-lon grid and will regrid a GEOS-IT Cubed-Sphere file to Lat-Lon. The convert_tool.py script must be run with the Python you installed as part of the prerequisites and don't forget to make sure it is in your path! The regridding_tool.py script has the following options:

• -n, --input_file The input Cubed-Sphere file
• -o, --output_file The output lat-lon file name
• -i, --im_world The number of longitudinal points in the output lat-lon grid
• -j, --jm_world The number of latitudinal points in the output lat-lon grid
• -d, --dateline Either DC or DE, this controls whether the center or edge of the grid is on the dateline
• -p, --pole Either PC or PE, this controls either the center of the pole point is at the pole or half a delta below the pole
• -m, --method optional, default bilinear, other options are conserve, patch, conserve2nd, neareststod, this is the regrid method used by ESMF_RegridWeightGen and information about them can be found in the ESMF documentation: https://earthsystemmodeling.org/docs/nightly/develop/ESMF_refdoc/node9.html#SECTION090149000000000000000
• --grid_dir optional, specify a path where the cubed-sphere grid descriptor and weights will be generated and stored. If the files are already found will not recompute. Recommended to save time when regridding multiple files. If this argument is not present, the files will be created and then deleted in the current directory
• --num_tasks optional, default 1, if on a platform that allows it (i.e. a compute node as NCCS or NAS) use more than 1 mpi task when generating the weights.
• -v, --vars optional, comma separated list of variables to regrid if you do not with to regrid all of the variables in the input file, no spaces in the list

For example, to regrid to a 360x180 PE,DE grid you would run the following command:

./convert_tool.py -n path_to_input_file -o output_file -i 360 -j 180 -d DE -p PE

If you intend to regrid multiple files to the same Lat-Lon resolution you should make use of the --grid_dir option. This will store the interpolation weights in the specified directory and will reuse them on subsequent runs if they are found here. When regridding multiple files saving and reusing the interpolation weights is highly recommended for efficiency as depending on the size of the grids involved computing the interpolation weights could be expensive.