Skip to content

Commit

Permalink
Split tutorial across multiple pages
Browse files Browse the repository at this point in the history
  • Loading branch information
shoyer committed Aug 4, 2021
1 parent 4fb8066 commit f614e5b
Show file tree
Hide file tree
Showing 8 changed files with 1,014 additions and 651 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,5 @@ dist
docs/.ipynb_checkpoints
docs/_build
docs/_autosummary
docs/my-data.zarr
__pycache__
59 changes: 59 additions & 0 deletions docs/aggregation.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "aaf2f2cb",
"metadata": {},
"source": [
"# Aggregation\n",
"\n",
"Xarray-Beam can perfrom distributed data aggregation in the \"map-reduce\" model. This currently only includes `Mean`, but we would welcome other contributions."
]
},
{
"cell_type": "markdown",
"id": "4b77b953",
"metadata": {},
"source": [
"Note that {py:class}`Mean.Globally <xarray_beam.Mean.Globally>` and {py:class}`Mean.PerKey <xarray_beam.Mean.PerKey>` are modelled off of [`beam.Mean`](https://beam.apache.org/documentation/transforms/python/aggregation/mean/) rather than {py:meth}`xarray.Dataset.mean`: they compute averages over sequences of `xarray.Dataset` objects, rather than calculating an average over a dimension."
]
},
{
"cell_type": "markdown",
"id": "37193539",
"metadata": {},
"source": [
"If you want to average over dimension, you'll currently need to do that aggregation yourself, e.g., by averaging inside each chunk before combining the data."
]
},
{
"cell_type": "markdown",
"id": "c9db1627",
"metadata": {},
"source": [
"TODO(shoyer): add code examples! For now, look at [ERA5 climatology](https://github.com/google/xarray-beam/blob/main/examples/era5_climatology.py)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading

0 comments on commit f614e5b

Please sign in to comment.