Skip to content

Commit

Permalink
Merge branch 'main' of github.com:saschahofmann/xclim into fix-Ourano…
Browse files Browse the repository at this point in the history
  • Loading branch information
saschahofmann committed Jan 14, 2025
2 parents 1c7d023 + eda9f51 commit d130da0
Show file tree
Hide file tree
Showing 13 changed files with 208 additions and 49 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/add-to-project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
- name: Generate App Token
id: token_generator
uses: actions/create-github-app-token@5d869da34e18e7287c1daad50e0b8ea0f506ce69 # v1.11.0
uses: actions/create-github-app-token@c1a285145b9d317df6ced56c09f525b5c2b6f755 # v1.11.1
with:
app-id: ${{ secrets.OURANOS_HELPER_BOT_ID }}
private-key: ${{ secrets.OURANOS_HELPER_BOT_KEY }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/bump-version.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
pypi.org:443
- name: Generate App Token
id: token_generator
uses: actions/create-github-app-token@5d869da34e18e7287c1daad50e0b8ea0f506ce69 # v1.11.0
uses: actions/create-github-app-token@c1a285145b9d317df6ced56c09f525b5c2b6f755 # v1.11.1
with:
app-id: ${{ secrets.OURANOS_HELPER_BOT_ID }}
private-key: ${{ secrets.OURANOS_HELPER_BOT_KEY }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/scorecard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ jobs:
# Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
# format to the repository Actions tab.
- name: Upload Artifact
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
with:
name: SARIF file
path: results.sarif
Expand Down
12 changes: 6 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ default_language_version:

repos:
- repo: https://github.com/asottile/pyupgrade
rev: v3.19.0
rev: v3.19.1
hooks:
- id: pyupgrade
args: ['--py310-plus']
Expand Down Expand Up @@ -37,12 +37,12 @@ repos:
hooks:
- id: black
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.1
rev: v0.8.6
hooks:
- id: ruff
args: [ '--fix', '--show-fixes' ]
- repo: https://github.com/pylint-dev/pylint
rev: v3.3.2
rev: v3.3.3
hooks:
- id: pylint
args: [ '--rcfile=.pylintrc.toml', '--errors-only', '--jobs=0', '--disable=import-error' ]
Expand All @@ -53,7 +53,7 @@ repos:
additional_dependencies: [ 'flake8-rst-docstrings ']
args: [ '--config=.flake8' ]
- repo: https://github.com/jendrikseipp/vulture
rev: 'v2.13'
rev: 'v2.14'
hooks:
- id: vulture
- repo: https://github.com/nbQA-dev/nbQA
Expand Down Expand Up @@ -82,7 +82,7 @@ repos:
- id: rst-inline-touching-normal
- id: text-unicode-replacement-char
- repo: https://github.com/executablebooks/mdformat
rev: 0.7.19
rev: 0.7.21
hooks:
- id: mdformat
exclude: '.github/\w+.md|.github/publish-mastodon-template.md|docs/paper/paper.md'
Expand All @@ -106,7 +106,7 @@ repos:
# Exclude the missing submodule from the xclim.core, see:
exclude: ^docs/|^tests/|^src/xclim/sdba|^src/xclim/core/missing.py
- repo: https://github.com/gitleaks/gitleaks
rev: v8.21.2
rev: v8.22.1
hooks:
- id: gitleaks
- repo: https://github.com/python-jsonschema/check-jsonschema
Expand Down
7 changes: 6 additions & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,12 @@ Changelog

v0.55.0 (unreleased)
--------------------
Contributors to this version: Sascha Hofmann (:user:`saschahofmann`).
Contributors to this version: Juliette Lavoie (:user:`juliettelavoie`), Sascha Hofmann (:user:`saschahofmann`).


New features and enhancements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* New function ``ensemble.partition.general_partition`` (:pull:`2035`)

Bug fixes
^^^^^^^^^
Expand Down
6 changes: 3 additions & 3 deletions CI/requirements_ci.in
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
bump-my-version==0.28.1
deptry==0.21.1
bump-my-version==0.29.0
deptry==0.21.2
flit==3.10.1
pip==24.3.1
pylint==3.3.2
pylint==3.3.3
tox==4.23.2
tox-gh==1.4.4
54 changes: 29 additions & 25 deletions CI/requirements_ci.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,17 @@ annotated-types==0.7.0 \
--hash=sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53 \
--hash=sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89
# via pydantic
astroid==3.3.6 \
--hash=sha256:6aaea045f938c735ead292204afdb977a36e989522b7833ef6fea94de743f442 \
--hash=sha256:db676dc4f3ae6bfe31cda227dc60e03438378d7a896aec57422c95634e8d722f
astroid==3.3.8 \
--hash=sha256:187ccc0c248bfbba564826c26f070494f7bc964fd286b6d9fff4420e55de828c \
--hash=sha256:a88c7994f914a4ea8572fac479459f4955eeccc877be3f2d959a33273b0cf40b
# via pylint
bracex==2.4 \
--hash=sha256:a27eaf1df42cf561fed58b7a8f3fdf129d1ea16a81e1fadd1d17989bc6384beb \
--hash=sha256:efdc71eff95eaff5e0f8cfebe7d01adf2c8637c8c92edaf63ef348c241a82418
# via wcmatch
bump-my-version==0.28.1 \
--hash=sha256:df7fdb02a1b43c122a6714df6d1fe4efc7a1220b5638ca5a0eb3018813c1b222 \
--hash=sha256:e608def5191baf505b6cde88bd679a0a95fc4cfeace4247adb60ac0f8a7e57ee
bump-my-version==0.29.0 \
--hash=sha256:6566ab25bd3eeeec109f4ac7e4464227a3ac1fd57f847d259a24800423cd9037 \
--hash=sha256:e4149ed63b4772f5868b3fcabb8fa5e1191b8abae6d35effd0be980d4b0f55e3
# via -r CI/requirements_ci.in
cachetools==5.5.0 \
--hash=sha256:02134e8439cdc2ffb62023ce1debca2944c3f289d66bb17ead3ab3dede74b292 \
Expand Down Expand Up @@ -124,9 +124,9 @@ charset-normalizer==3.3.2 \
--hash=sha256:fd1abc0d89e30cc4e02e4064dc67fcc51bd941eb395c502aac3ec19fab46b519 \
--hash=sha256:ff8fa367d09b717b2a17a052544193ad76cd49979c805768879cb63d9ca50561
# via requests
click==8.1.7 \
--hash=sha256:ae74fb96c20a0277a1d615f1e4d73c8414f5a98db8b799a7931d1582f3390c28 \
--hash=sha256:ca9853ad459e787e2192211578cc907e7594e294c7ccc834310722b41b9ca6de
click==8.1.8 \
--hash=sha256:63c132bbbed01578a06712a2d1f497bb62d9c1c0d329b7903a866228027263b2 \
--hash=sha256:ed53c9d8990d83c2a27deae68e4ee337473f6330c040a31d4225c9574d16096a
# via
# bump-my-version
# deptry
Expand All @@ -135,19 +135,23 @@ colorama==0.4.6 \
--hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \
--hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6
# via tox
deptry==0.21.1 \
--hash=sha256:091288cad2bd6029995d2e700e965cd574079365807f202ee232e4be0a571f43 \
--hash=sha256:145a172ea608bb86dd93a9d14f7d45ed8649a36d7f685ea725e0348cbf562f10 \
--hash=sha256:1adf29a5aa1d33d9e1140b9235b212d9753278604b4389b2186f638692e29876 \
--hash=sha256:4afef1c5eb0b48ebc31de2437b460df0363cb99722252b7faf7fa6f43e10cbcd \
--hash=sha256:4b53089c22d18076935a3e9e6325566fa712cd9b89fe602978a8e85f0f4209bf \
--hash=sha256:60332b8d58d6584b340511a4e1b694048499f273d69eaea413631b2e8bc186ff \
--hash=sha256:79593d7631cdbbc39d76503e3af80e46d8b4873e915b85c1567a04c81e8a17d5 \
--hash=sha256:98075550540c6b45f57abdfc453900bd2a179dc495d986ccc0757a813ee55103 \
--hash=sha256:981a28e1feeaad82f07a6e3c8d7842c5f6eae3807dc13b24d453a20cd0a42a72 \
--hash=sha256:b5eae7afbcb9b7f6baa855b323e0da016a23f2a98d4b181dcfd2c71766512387 \
--hash=sha256:c31e1a66502e28870e1e0a679598462a6119f4bcb656786e63cb545328170a3f \
--hash=sha256:e487f520d4fbee513f4767ab98334a29d5d932f78eb413b64e27c977f2bf2756
deptry==0.21.2 \
--hash=sha256:019167b35301edd2bdd4719c8b8f44769be4507cb8a1cd46fff4393cdbe8d31b \
--hash=sha256:06d48e9fa460aad02f9e1b079d9f5a69d622d291b3a0525b722fc91c88032042 \
--hash=sha256:1012a88500f242489066f811f6ec0c93328d9340bbf0f87f0c7d2146054d197e \
--hash=sha256:186ddbc69c1f70e684e83e202795e1054d0c2dfc03b8acc077f65dc3b6a7f4ce \
--hash=sha256:3080bb88c16ebd35f59cba7688416115b7aaf4630dc5a051dff2649cbf129a1b \
--hash=sha256:3ef8aed33a2eac357f9565063bc1257bcefa03a37038299c08a4222e28f3cd34 \
--hash=sha256:4e870553c7a1fafcd99a83ba4137259525679eecabeff61bc669741efa201541 \
--hash=sha256:7479d3079be69c3bbf5913d8e21090749c1139ee91f81520ffce90b5322476b0 \
--hash=sha256:769bb658172586d1b03046bdc6b6c94f6a98ecfbac04ff7f77ec61768c75e1c2 \
--hash=sha256:917745db5f8295eb5048e43d9073a9a675ffdba865e9b294d2e7aa455730cb06 \
--hash=sha256:adb12d6678fb5dbd320a0a2e37881059d0a45bec6329df4250c977d803fe7f96 \
--hash=sha256:d76bbf48bd62ecc44ca3d414769bd4b7956598d23d9ccb42fd359b831a31cab2 \
--hash=sha256:d8add495f0dd19a38aa6d1e09b14b1441bca47c9d945bc7b322efb084313eea3 \
--hash=sha256:e3b9e0c5ee437240b65e61107b5777a12064f78f604bf9f181a96c9b56eb896d \
--hash=sha256:f3686e86ad7063b5a6e5253454f9d9e4a7a6b1511a99bd4306fda5424480be48 \
--hash=sha256:fb2f43747b58abeec01dc277ef22859342f3bca2ac677818c94940a009b436c0
# via -r CI/requirements_ci.in
dill==0.3.8 \
--hash=sha256:3ebe3c479ad625c4553aca177444d89b486b1d84982eeacded644afc0cf797ca \
Expand Down Expand Up @@ -323,9 +327,9 @@ pygments==2.18.0 \
--hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \
--hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a
# via rich
pylint==3.3.2 \
--hash=sha256:77f068c287d49b8683cd7c6e624243c74f92890f767f106ffa1ddf3c0a54cb7a \
--hash=sha256:9ec054ec992cd05ad30a6df1676229739a73f8feeabf3912c995d17601052b01
pylint==3.3.3 \
--hash=sha256:07c607523b17e6d16e2ae0d7ef59602e332caa762af64203c24b41c27139f36a \
--hash=sha256:26e271a2bc8bce0fc23833805a9076dd9b4d5194e2a02164942cb3cdc37b4183
# via -r CI/requirements_ci.in
pyproject-api==1.8.0 \
--hash=sha256:3d7d347a047afe796fd5d1885b1e391ba29be7169bd2f102fcd378f04273d228 \
Expand Down
1 change: 1 addition & 0 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,4 @@ dependencies:
- xdoctest >=1.1.5
- yamllint >=1.35.1
- pip >=24.2.0
- pygments <2.19 #FIXME: temporary fix, https://github.com/felix-hilden/sphinx-codeautolink/issues/153
9 changes: 5 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,11 @@ dev = [
# Dev tools and testing
"black[jupyter] ==24.10.0",
"blackdoc ==0.3.9",
"bump-my-version ==0.28.1",
"bump-my-version ==0.29.0",
"codespell ==2.3.0",
"coverage[toml] >=7.5.0",
"coveralls >=4.0.1", # coveralls is not yet compatible with Python 3.13
"deptry ==0.21.1",
"deptry ==0.21.2",
"flake8 >=7.1.1",
"flake8-rst-docstrings >=0.3.0",
"h5netcdf>=1.3.0",
Expand Down Expand Up @@ -111,7 +111,8 @@ docs = [
"sphinx-copybutton",
"sphinx-mdinclude",
"sphinxcontrib-bibtex",
"sphinxcontrib-svg2pdfconverter[Cairosvg]"
"sphinxcontrib-svg2pdfconverter[Cairosvg]",
"pygments <2.19" # FIXME: temporary fix, https://github.com/felix-hilden/sphinx-codeautolink/issues/153
]
extras = ["fastnanquantile >=0.0.2", "flox >=0.9", "POT >=0.9.4"]
all = ["xclim[dev]", "xclim[docs]", "xclim[extras]"]
Expand All @@ -138,7 +139,7 @@ target-version = [
]

[tool.bumpversion]
current_version = "0.54.0"
current_version = "0.54.1-dev.0"
commit = true
commit_args = "--no-verify --signoff"
tag = false
Expand Down
2 changes: 1 addition & 1 deletion src/xclim/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

__author__ = """Travis Logan"""
__email__ = "[email protected]"
__version__ = "0.54.0"
__version__ = "0.54.1-dev.0"


with _resources.as_file(_resources.files("xclim.data")) as _module_data:
Expand Down
1 change: 1 addition & 0 deletions src/xclim/ensembles/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
)
from xclim.ensembles._partitioning import (
fractional_uncertainty,
general_partition,
hawkins_sutton,
lafferty_sriver,
)
Expand Down
133 changes: 128 additions & 5 deletions src/xclim/ensembles/_partitioning.py
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,128 @@ def lafferty_sriver(
return g, uncertainty


def general_partition(
da: xr.DataArray,
sm: xr.DataArray | str = "poly",
var_first: list | None = None,
mean_first: list | None = None,
weights: list | None = None,
) -> tuple[xr.DataArray, xr.DataArray]:
"""
Return the mean and partitioned variance of an ensemble.
This is a general function that can be used to implemented methods from different papers.
The defaults are set to match the Lavoie et al. (2025, in preparation) method.
Parameters
----------
da : xr.DataArray
Time series with dimensions 'time', 'mean_first', and 'var_first'.
sm : xr.DataArray or {"poly"}
Smoothed time series over time, with the same dimensions as `da`.
If 'poly', this is estimated using a 4th-order polynomial.
It is also possible to pass a precomputed smoothed time series.
var_first : list of str
List of dimensions where the variance is computed first of the dimension,
followed by the mean over the other dimensions.
mean_first : list of str
List of dimensions where the mean over the other dimensions is computed first,
followed by the variance over the dimension.
weights : list of str
List of dimensions where the first operation is weighted.
Returns
-------
xr.DataArray, xr.DataArray
The mean relative to the baseline, and the components of variance of the
ensemble. These components are coordinates along the `uncertainty` dimension:
element of var_first, elements of mean_first and `total`.
Notes
-----
To prepare input data, make sure `da` has dimensions list in both var_first and
mean_first, as well as time.
e.g. `da.rename({"experiment": "scenario"})`.
To get the fraction of the total variance instead of the variance itself, call `fractional_uncertainty` on the
output.
"""
# set defaults
var_first = var_first or ["model", "reference", "adjustment"]
mean_first = mean_first or ["scenario"]
weights = weights or ["model", "reference", "adjustment"]

all_types = mean_first + var_first

if xr.infer_freq(da.time)[0] not in ["A", "Y"]:
raise ValueError("This algorithm expects annual time series.")

if not ({"time"} | set(all_types)).issubset(da.dims):
error_msg = f"DataArray dimensions should include {all_types} and time."
raise ValueError(error_msg)

if sm == "poly":
# Fit a 4th order polynomial
fit = da.polyfit(dim="time", deg=4, skipna=True)
sm = xr.polyval(coord=da.time, coeffs=fit.polyfit_coefficients).where(
da.notnull()
)
elif isinstance(sm, xr.DataArray):
sm = sm
else:
raise ValueError("sm should be 'poly' or a DataArray.")

# "Interannual variability is then estimated as the centered rolling 11-year variance of the difference
# between the extracted forced response and the raw outputs, averaged over all outputs."
# same as lafferty_sriver()
nv_u = (da - sm).rolling(time=11, center=True).var().mean(dim=all_types)

all_u = []
total = nv_u.copy()
for t in mean_first:
all_but_t = [x for x in all_types if x != t]
if t in weights:
tw = sm.count(t)
t_u = sm.mean(dim=all_but_t).weighted(tw).var(dim=t)

else:
t_u = sm.mean(dim=all_but_t).var(dim=t)
all_u.append(t_u)
total += t_u

for t in var_first:
all_but_t = [x for x in all_types if x != t]
if t in weights:
tw = sm.count(t)
t_u = sm.var(dim=t).weighted(tw).mean(dim=all_but_t)

else:
t_u = sm.var(dim=t).mean(dim=all_but_t)
all_u.append(t_u)
total += t_u

# Create output array with the uncertainty components
u = pd.Index([*all_types, "variability", "total"], name="uncertainty")
uncertainty = xr.concat([*all_u, nv_u, total], dim=u)

uncertainty.attrs["indicator_long_name"] = da.attrs.get("long_name", "unknown")
uncertainty.attrs["indicator_description"] = da.attrs.get("description", "unknown")
uncertainty.attrs["indicator_units"] = da.attrs.get("units", "unknown")
uncertainty.attrs["partition_fit"] = sm if isinstance(sm, str) else "unknown"
# Keep a trace of the elements for each uncertainty component
for t in all_types:
uncertainty.attrs[t] = da[t].values

# Mean projection:
# This is not part of the original algorithm,
# but we want all partition algos to have similar outputs.
g = sm.mean(dim=all_types[0])
for dim in all_types[1:]:
g = g.mean(dim=dim)

return g, uncertainty


def fractional_uncertainty(u: xr.DataArray) -> xr.DataArray:
"""
Return the fractional uncertainty.
Expand All @@ -316,8 +438,9 @@ def fractional_uncertainty(u: xr.DataArray) -> xr.DataArray:
xr.DataArray
Fractional, or relative uncertainty with respect to the total uncertainty.
"""
uncertainty = u / u.sel(uncertainty="total") * 100
uncertainty.attrs.update(u.attrs)
uncertainty.attrs["long_name"] = "Fraction of total variance"
uncertainty.attrs["units"] = "%"
return uncertainty
with xr.set_options(keep_attrs=True):
uncertainty = u / u.sel(uncertainty="total") * 100
uncertainty.attrs.update(u.attrs)
uncertainty.attrs["long_name"] = "Fraction of total variance"
uncertainty.attrs["units"] = "%"
return uncertainty
Loading

0 comments on commit d130da0

Please sign in to comment.