Skip to content

Commit

Permalink
ENH: Update the reading of geo-referenced input data
Browse files Browse the repository at this point in the history
The aim of this commit is to address the problems associated with reading geo-referenced data (atmospheric and physiographic data).
In particular, certain specific cases were not fully handled by smash:
- Different spatial resolution
- Overlap missmatch
- Mesh extent greater than data extent.
This commit improves these 3 aspects by returning a warning if one of these cases occurs on one of the files read.
For the resolution, the file is resampled using nearest neighbour algorithm. For overlap missmatch, the reading window is shifted to overlap on the
nearest cell. And, if the mesh extension is too large, the code no longer returns an error but fills the array with no data in the part whose extent
is not covered by the data.

In addition, when generating a mesh with a bounding box, if the given bounding box does not overlap with the flow directions, it is padded.

Finally, a series of tests on the various possible warnings and some documentation has been added
  • Loading branch information
inoelloc committed Apr 12, 2024
1 parent 24b955d commit 6b92837
Show file tree
Hide file tree
Showing 9 changed files with 418 additions and 86 deletions.
26 changes: 24 additions & 2 deletions doc/source/release/1.0.0-notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ Below is the table of equivalence between the two versions:
* - ``gr-a``
- No equivalence
* - ``gr-b``
- ``zero`` - ``gr4`` - ``lr``
- No equivalence
* - ``gr-c``
- No equivalence
* - ``gr-d``
Expand All @@ -161,6 +161,25 @@ file is clipped by taking as its maximum extension the number of cells required
rectangular catchment one cell wide. In this way, it is possible to create a mesh of a French catchment using
worldwide flow direction.

Reading of atmospheric and physiograhic data
********************************************

The reading of geo-referenced data (i.e. precipitation, snow, descriptor, etc) has been improved to manage:

- ``Resolution missmatch``
Nearest-Neighbour resample algorithm is used to match the resolution of the data and the resolution of the
mesh.

- ``Overlap missmatch``
The reading window is correctly shifted to overlap the data and the mesh on the nearest overlapping cell.

- ``Out Of Bound``
In the previous version, when the mesh extent was larger than the data extent a not a very informative
error was returned. It is now possible to read data whose extent partially include the mesh extent. The
corresponding out of bound extent will be filled in with no data.

A warning is returned if one of the 3 cases below is encountered when reading the data.

Mean atmospheric data
*********************

Expand Down Expand Up @@ -334,7 +353,10 @@ The precipitation partitioning can be selected using the ``prcp_partitioning`` s
New hydrological modules
************************

3 new hydrological modules have been introduced:
4 new hydrological modules have been introduced:

- ``gr4``
This hydrological module is derived from the GR4 model :cite:p:`perrin2003improvement`.

- ``gr5``
This hydrological module is derived from the GR5 model :cite:p:`LeMoine_2008`.
Expand Down
74 changes: 46 additions & 28 deletions doc/source/user_guide/others/input_data_convention.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,39 @@ Input Data Convention
=====================

The aim of this section is to describe the conventions that apply to all input data used in `smash`
(i.e. precipitation, observed discharge, descriptor, etc)
(i.e. observed discharge, precipitation, descriptor, etc)

Observed discharge
------------------

The observed discharge for one catchment is read from a ``.csv`` file with the following structure:

.. csv-table:: V3524010.csv
:align: center
:header: "200601010000"
:width: 50

-99.000
-99.000
...
1.180
1.185

It is a single-column ``.csv`` file containing the observed discharge values in m\ :sup:`3` \/s (negative values in the series will be interpreted
as no-data) and whose header is the first time step of the chronicle. The name of the file, for any catchment, must contains the code of the
gauge which is filled in the ``mesh`` (see the `smash.factory.generate_mesh` method).

.. note::
The time step of the header does not have to match the first simulation time step. `smash` manages to read the corresponding lines
from the ``setup`` variables, ``start_time``, ``end_time`` and ``dt``.


Precipitation
-------------

The precipitation files must be stored for each time step of the simulation in ``tif`` format. For one time step, `smash` will recursively
search in the ``prcp_directory``, a file with the following name structure: ``*%Y%m%d%H%M*.tif`` (``*`` means that we can match any character).
An example of file name in tif format for the date 2014-09-15 00:00: ``prcp_201409150000.tif``. The spatial resolution and the projection
between the precipitation files and the flow direction file must be **identical**.
An example of file name in tif format for the date 2014-09-15 00:00: ``prcp_201409150000.tif``.

.. note::
``%Y%m%d%H%M`` is a unique key, the ``prcp_directory`` (and all subdirectories) can not contains files with similar dates.
Expand Down Expand Up @@ -60,30 +84,6 @@ between the temperature files and the flow direction file must be **identical**.
.. note::
``%Y%m%d%H%M`` is a unique key, the ``temp_directory`` (and all subdirectories) can not contains files with similar dates.

Observed discharge
------------------

The observed discharge for one catchment is read from a ``.csv`` file with the following structure:

.. csv-table:: V3524010.csv
:align: center
:header: "200601010000"
:width: 50

-99.000
-99.000
...
1.180
1.185

It is a single-column ``.csv`` file containing the observed discharge values in m\ :sup:`3` \/s (negative values in the series will be interpreted
as no-data) and whose header is the first time step of the chronicle. The name of the file, for any catchment, must contains the code of the
gauge which is filled in the ``mesh`` (see the `smash.factory.generate_mesh` method).

.. note::
The time step of the header does not have to match the first simulation time step. `smash` manages to read the corresponding lines
from the ``setup`` variables, ``start_time``, ``end_time`` and ``dt``.

Descriptor
----------

Expand All @@ -93,9 +93,27 @@ An example of file name in tif format for the slope descriptor: ``slope.tif``. T
between the decriptor files and the flow direction file must be **identical**.

.. note::

``descriptor_name`` is a unique key, the ``descriptor_directory`` (and all subdirectories) can not contains files with similar decriptor name.

.. warning::
There are 4 possible warnings when reading geo-referenced data (i.e. precipitation, descriptor, etc):

- ``Missing Warning``
A file (or more) is missing. It will be interpreted as no data.

- ``Resolution Warning``
A file (or more) has a spatial resolution different from the mesh resolution (i.e. the flow direction resolution).
It will be resampled using a Nearest Neighbour algorithm.

- ``Overlap Warning``
A file (or more) has an origin that does not overlap with the mesh origin (i.e. the flow direction origin).
The reading window is shifted towards the nearest overlapping cell.

- ``Out Of Bound Warning``
A file (or more) has an extent that does not include, partially or totally, the mesh extent.
It will be interpreted as no data where the mesh extent is out of bound.


Directory structure
-------------------

Expand Down
Loading

0 comments on commit 6b92837

Please sign in to comment.