Skip to content

Latest commit

 

History

History
89 lines (66 loc) · 2.91 KB

README_using_fre-cli.md

File metadata and controls

89 lines (66 loc) · 2.91 KB

note these instructions will be/are from https://github.com/NOAA-GFDL/fre-cli/tree/main/fre/pp

Instructions to postprocess FMS history output on PP/AN or gaea with fre-cli

  1. Checkout postprocessing workflow template This will clone the postprocessing repository into /home/$USER/cylc-src/EXPNAME__PLATFORM__TARGET.
module load fre/canopy
fre pp checkout -e EXPNAME -p PLATFORM -t TARGET
  1. Configure pp template with either XML or pp.yaml
fre pp configure-xml -e EXPNAME -p PLATFORM -t TARGET -x XML

or

fre pp configure-yaml -e EXPNAME -p PLATFORM -t TARGET -y YAML
  1. (OPTIONAL BUT RECCOMENDED) Create history-manifest for config validation

Create a history-manifest of a single tar file archive first for use in the validation. This list represents the available source files within the history tar archives, and enables the validation procedure to catch a wider variety of potential errors. This can be done like so-

tar -tf /archive/$USER/path/to/history/files/YYYYMMDD.nc.tar | grep -v tile[2-6] | sort > /home/$USER/cylc-src/EXPNAME__PLATFORM__TARGET/history-manifest
  1. Validate the configuration
fre pp validate -e EXPNAME -p PLATFORM -t TARGET

Warnings related to directories are probably valid and should be fixed in rose-suite.conf, or created as necessary via mkdir.

If you are running postprocessing gaea, you'll need to change the SITE variable in rose-suite.conf from ppan to gaea.

  1. Install the workflow
fre pp install -e EXPNAME -p PLATFORM -t TARGET

If you are attempting this on gaea, you'll need to make two one-time changes before installing.

  • Currently, cylc, rose, and isodatetime must be in your PATH for new shells. One approach to do this is to symlink the fms-user-installed fre-cli cylc/rose/isodatetime scripts into your local ~/bin directory, and then add that ~/bin directory to your PATH in your .bashrc or .cshrc. (If you don't do this, Cylc tasks will fail complaining those 3 tools are not available.)
cd ~/bin
ln -s /ncrc/home2/Flexible.Modeling.System/conda/envs/fre-cli/bin/{cylc,rose,isodatetime} .
echo 'setenv PATH ${PATH}:~/bin' >> ~/.cshrc
  • Currently, the cylc available on gaea (through module load cylc or the PATH trick above) does not include any global configuration, so you'll need to create a file ~/.cylc/flow/global.cylc that contains the following. If you don't do this, Cylc will use your home directory for the scratch space and rapidly fill your quota.)
[install]
    [[symlink dirs]]
        [[[localhost]]]
            run = /gpfs/f5/scratch/gfdl_f/$USER
  1. Run the workflow
fre pp run -e EXPNAME -p PLATFORM -t TARGET
  1. Report status of workflow progress
fre pp status -e EXPNAME -p PLATFORM -t TARGET
  1. Launch GUI
TODO: fre pp gui?

The full GUI can be launched on jhan or jhanbigmem (an107 or an201).

cylc gui --ip=`hostname -f` --port=`jhp 1` --no-browser