Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes from integration with SRW's make_orog #603

Merged
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/sections/user_guide/cli/drivers/filter_topo/help.out
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ Optional arguments:

Positional arguments:
TASK
filtered_output_file
The filtered output file staged from raw input
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
input_grid_file
The input grid file
namelist_file
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"config": {
"additionalProperties": false,
"properties": {
"input_grid_file": {
"filtered_orog": {
"type": "string"
...
"rundir"
Expand Down
2 changes: 2 additions & 0 deletions docs/sections/user_guide/cli/drivers/orog_gsl/help.out
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ Optional arguments:

Positional arguments:
TASK
input_config_file
The input config file
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
input_grid_file
The input grid file
provisioned_rundir
Expand Down
2 changes: 2 additions & 0 deletions docs/sections/user_guide/cli/drivers/shave/help.out
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ Optional arguments:

Positional arguments:
TASK
input_config_file
The input config file
provisioned_rundir
Run directory provisioned with all required content
run
Expand Down
8 changes: 8 additions & 0 deletions docs/sections/user_guide/yaml/components/filter_topo.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,18 @@ config:

Configuration parameters for the ``orog_gsl`` component.

**filtered_orog:**

Name of the filtered output file.

christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
**input_grid_file:**

Path to the tiled input grid file.

**input_raw_orog:**

Path to the raw orography file. The output of the ``orog`` driver.
maddenp-noaa marked this conversation as resolved.
Show resolved Hide resolved

namelist:
^^^^^^^^^

Expand Down
6 changes: 5 additions & 1 deletion docs/sections/user_guide/yaml/components/shave.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Describes the required parameters to run a ``shave`` configuration.

Name of the grid file with extra points to be shaved.

**nh4:**
**nhalo:**
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved

The number of halo rows/columns.

Expand All @@ -39,6 +39,10 @@ Describes the required parameters to run a ``shave`` configuration.

The j/y dimensions of the compute domain (not including halo)

**output_grid_file:**

The path to the output file.
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved

rundir:
^^^^^^^

Expand Down
2 changes: 2 additions & 0 deletions docs/shared/filter_topo.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
filter_topo:
config:
filtered_orog: C403_filtered_orog.tile7.nc
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
input_grid_file: /path/to/C403_grid.tile7.halo6.nc
input_raw_orog: /path/to/out.oro.nc
execution:
batchargs:
cores: 1
Expand Down
12 changes: 6 additions & 6 deletions docs/shared/orog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ orog:
cores: 1
walltime: 00:05:00
executable: /path/to/orog
files_to_link:
fort.15: /path/to/fix/thirty.second.antarctic.new.bin
landcover30.fixed: /path/to/fix/landcover30.fixed
fort.235: /path/to/fix/gmted2010.30sec.int
grid_file: /path/to/netcdf/grid/file
rundir: /path/to/run/dir
files_to_link:
fort.15: /path/to/fix/thirty.second.antarctic.new.bin
landcover30.fixed: /path/to/fix/landcover30.fixed
fort.235: /path/to/fix/gmted2010.30sec.int
grid_file: /path/to/netcdf/grid/file
rundir: /path/to/run/dir
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
platform:
account: me
scheduler: slurm
3 changes: 2 additions & 1 deletion docs/shared/shave.yaml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
shave:
config:
input_grid_file: /path/to/input/grid/file
nh4: 1
nhalo: 0
nx: 214
ny: 128
output_grid_file: /path/to/C403_oro_data.tile7.halo0.nc
execution:
batchargs:
cores: 1
Expand Down
16 changes: 14 additions & 2 deletions src/uwtools/drivers/filter_topo.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from uwtools.drivers.driver import DriverTimeInvariant
from uwtools.drivers.support import set_driver_docstring
from uwtools.strings import STR
from uwtools.utils.tasks import symlink
from uwtools.utils.tasks import filecopy, symlink


class FilterTopo(DriverTimeInvariant):
Expand All @@ -27,10 +27,21 @@ def input_grid_file(self):
"""
src = Path(self.config["config"]["input_grid_file"])
dst = Path(self.config[STR.rundir], src.name)
yield self.taskname("Input grid")
yield self.taskname(f"Input grid {str(src)}")
yield asset(dst, dst.is_file)
yield symlink(target=src, linkname=dst)

@task
def filtered_output_file(self):
"""
The filtered output file staged from raw input.
"""
src = Path(self.config["config"]["input_raw_orog"])
dst = self.rundir / self.config["config"]["filtered_orog"]
yield self.taskname(f"Raw orog input {str(dst)}")
yield asset(dst, dst.is_file)
yield filecopy(src=src, dst=dst)
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved

@task
def namelist_file(self):
"""
Expand All @@ -56,6 +67,7 @@ def provisioned_rundir(self):
yield self.taskname("provisioned run directory")
yield [
self.input_grid_file(),
self.filtered_output_file(),
self.namelist_file(),
self.runscript(),
]
Expand Down
23 changes: 20 additions & 3 deletions src/uwtools/drivers/orog.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def grid_file(self):
The input grid file.
"""
grid_file = Path(self.config["grid_file"])
yield self.taskname("Input grid file")
yield self.taskname(f"Input grid file {grid_file}")
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
yield asset(grid_file, grid_file.is_file) if str(grid_file) != "none" else None

@task
Expand All @@ -62,8 +62,9 @@ def input_config_file(self):
"blat",
]
inputs = " ".join([str(inputs[i]) for i in ordered_entries])
outgrid = self.config["grid_file"]
orogfile = self.config.get("orog_file")
outgrid = "'{}'".format(self.config["grid_file"])
if orogfile := self.config.get("orog_file"):
orogfile = "'{}'".format(orogfile)
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
mask_only = ".true." if self.config.get("mask") else ".false."
merge_file = self.config.get("merge", "none") # string none is intentional
content = [i for i in [inputs, outgrid, orogfile, mask_only, merge_file] if i is not None]
Expand All @@ -82,6 +83,22 @@ def provisioned_rundir(self):
self.runscript(),
]

@task
def runscript(self):
"""
The runscript.
"""
path = self._runscript_path
yield self.taskname(path.name)
yield asset(path, path.is_file)
yield None
envvars = {
"KMP_AFFINITY": "disabled",
"OMP_NUM_THREADS": self.config.get(STR.execution, {}).get(STR.threads, 1),
"OMP_STACKSIZE": "2048m",
}
self._write_runscript(path=path, envvars=envvars)
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved

# Public helper methods

@classmethod
Expand Down
24 changes: 22 additions & 2 deletions src/uwtools/drivers/orog_gsl.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from uwtools.drivers.driver import DriverTimeInvariant
from uwtools.drivers.support import set_driver_docstring
from uwtools.strings import STR
from uwtools.utils.file import writable
from uwtools.utils.tasks import symlink


Expand All @@ -19,6 +20,19 @@ class OrogGSL(DriverTimeInvariant):

# Workflow tasks

@task
def input_config_file(self):
"""
The input config file.
"""
path = self._input_config_path
yield self.taskname(str(path))
yield asset(path, path.is_file)
yield None
inputs = [str(self.config["config"][k]) for k in ("tile", "resolution", "halo")]
with writable(path) as f:
print("\n".join(inputs), file=f)
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved

@task
def input_grid_file(self):
"""
Expand All @@ -40,6 +54,7 @@ def provisioned_rundir(self):
"""
yield self.taskname("provisioned run directory")
yield [
self.input_config_file(),
self.input_grid_file(),
self.runscript(),
self.topo_data_2p5m(),
Expand Down Expand Up @@ -80,15 +95,20 @@ def driver_name(cls) -> str:
return STR.oroggsl

# Private helper methods
@property
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
def _input_config_path(self) -> Path:
"""
Path to the input config file.
"""
return self.rundir / "orog_gsl.cfg"

@property
def _runcmd(self):
"""
The full command-line component invocation.
"""
inputs = [str(self.config["config"][k]) for k in ("tile", "resolution", "halo")]
executable = self.config[STR.execution][STR.executable]
return "echo '%s' | %s" % ("\n".join(inputs), executable)
return "%s < %s" % (executable, self._input_config_path.name)
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved


set_driver_docstring(OrogGSL)
42 changes: 34 additions & 8 deletions src/uwtools/drivers/shave.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,14 @@
A driver for shave.
"""

from iotaa import tasks
from pathlib import Path

from iotaa import asset, task, tasks

from uwtools.drivers.driver import DriverTimeInvariant
from uwtools.drivers.support import set_driver_docstring
from uwtools.strings import STR
from uwtools.utils.file import writable


class Shave(DriverTimeInvariant):
Expand All @@ -16,13 +19,34 @@ class Shave(DriverTimeInvariant):

# Workflow tasks

@task
def input_config_file(self):
"""
The input config file.
"""
path = self._input_config_path
yield self.taskname(str(path))
yield asset(path, path.is_file)
config = self.config["config"]
input_file = Path(config["input_grid_file"])
yield asset(input_file, input_file.is_file)
flags = [
config[key] for key in ["nx", "ny", "nhalo", "input_grid_file", "output_grid_file"]
]
content = "{} {} {} '{}' '{}'".format(*flags)
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
with writable(path) as f:
print(content, file=f)
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved

@tasks
def provisioned_rundir(self):
"""
Run directory provisioned with all required content.
"""
yield self.taskname("provisioned run directory")
yield self.runscript()
yield [
self.input_config_file(),
self.runscript(),
]

# Public helper methods

Expand All @@ -35,18 +59,20 @@ def driver_name(cls) -> str:

# Private helper methods

@property
def _input_config_path(self) -> Path:
"""
Path to the input config file.
"""
return self.rundir / "shave.cfg"

@property
def _runcmd(self):
"""
The full command-line component invocation.
"""
executable = self.config[STR.execution][STR.executable]
config = self.config["config"]
input_file = config["input_grid_file"]
output_file = input_file.replace(".nc", "_NH0.nc")
flags = [config[key] for key in ["nx", "ny", "nh4", "input_grid_file"]]
flags.append(output_file)
return f"{executable} {' '.join(str(flag) for flag in flags)}"
return "%s < %s" % (executable, self._input_config_path.name)


set_driver_docstring(Shave)
3 changes: 3 additions & 0 deletions src/uwtools/resources/jsonschema/execution-serial.jsonschema
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
},
"executable": {
"type": "string"
},
"threads": {
"type": "integer"
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
}
},
"required": [
Expand Down
10 changes: 9 additions & 1 deletion src/uwtools/resources/jsonschema/filter-topo.jsonschema
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,20 @@
"config": {
"additionalProperties": false,
"properties": {
"filtered_orog": {
"type": "string"
},
"input_grid_file": {
"type": "string"
},
"input_raw_orog": {
"type": "string"
}
},
"required": [
"input_grid_file"
"filtered_orog",
"input_grid_file",
"input_raw_orog"
]
},
"execution": {
Expand Down
12 changes: 8 additions & 4 deletions src/uwtools/resources/jsonschema/shave.jsonschema
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
"input_grid_file": {
"type": "string"
},
"nh4": {
"minimum": 1,
"nhalo": {
"minimum": 0,
christinaholtNOAA marked this conversation as resolved.
Show resolved Hide resolved
"type": "integer"
},
"nx": {
Expand All @@ -20,13 +20,17 @@
"ny": {
"minimum": 1,
"type": "integer"
},
"output_grid_file": {
"type": "string"
}
},
"required": [
"input_grid_file",
"nh4",
"nhalo",
"nx",
"ny"
"ny",
"output_grid_file"
]
},
"execution": {
Expand Down
Loading
Loading