ML XGBoost Classification

.github/workflows/docs.yml

@@ -19,7 +19,7 @@ jobs:
       - run: |
           npm install
           npm run generate
-        working-directory: tests
+        working-directory: dev
       - name: clone gh-pages and clean-up
         if: ${{ env.GITHUB_REF_SLUG == 'master' }}
         run: |
@@ -31,8 +31,8 @@ jobs:
         if: ${{ env.GITHUB_REF_SLUG != 'master' }}
         run: mkdir gh-pages
       - run: |
-          cp tests/docs.html index.html
-          cp tests/processes.json processes.json
+          cp dev/docs.html index.html
+          cp dev/processes.json processes.json
           rsync -vrm --include='*.json' --include='*.html' --include='meta/***' --include='proposals/***' --exclude='*' . gh-pages
       - name: deploy to root (master)
         uses: peaceiris/actions-gh-pages@v3

.github/workflows/tests.yml

@@ -8,8 +8,8 @@ jobs:
         with:
           node-version: 'lts/*'
       - uses: actions/checkout@v3
-      - name: Run tests
+      - name: Run linter
         run: |
           npm install
-          npm run test
-        working-directory: tests
+          npm test
+        working-directory: dev

CHANGELOG.md

@@ -6,8 +6,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased / Draft
 
+### Changed
+
+- `clip`: Throw an exception if min > max [#472](https://github.com/Open-EO/openeo-processes/issues/472)
+
+### Fixed
+
+- `between`: Clarify that `null` is passed through.
+- `eq` and `neq`: Explicitly set the minimum value for the `delta` parameter.
+- `filter_bbox`, `load_collection`, `load_stac`: Clarified that the bounding box is reprojected to the CRS of the spatial data cube dimensions if required.
+- `filter_spatial`: Clarified that masking is applied using the given geometries. [#469](https://github.com/Open-EO/openeo-processes/issues/469)
+- `sqrt`: Clarified that NaN is returned for negative numbers.
+
 ## [2.0.0-rc.1] - 2023-05-25
 
+### Fixed
+
+- `array_append`: Added `number` type for labels to be consistent with other processes. Default to numerical index instead of string. Clarify that the `label` parameter only applies to labeled arrays.
+
 ### Added
 
 - New processes in proposal state:
@@ -17,6 +33,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     - `flatten_dimensions`
     - `load_geojson`
     - `load_url`
+    - `ml_fit_class_xgboost`
     - `unflatten_dimension`
     - `vector_buffer`
     - `vector_reproject`

README.md

@@ -35,7 +35,7 @@ This repository contains a set of files formally describing the openEO Processes
 * [implementation.md](meta/implementation.md) in the `meta` folder provide some additional implementation details for back-ends. For back-end implementors, it's highly recommended to read them.
 * [subtype-schemas.json](meta/subtype-schemas.json) in the `meta` folder defines common data types (`subtype`s) for JSON Schema used in openEO processes.
 * Previously, an `examples` folder contained examples of user-defined processes. These have been migrated to the [openEO Community Examples](https://github.com/Open-EO/openeo-community-examples/tree/main/processes) repository.
-* The [`tests`](tests/) folder can be used to test the process specification for validity and consistent "style". It also allows rendering the processes in a web browser. Check the [tests documentation](tests/README.md) for details.
+* The [`dev`](dev/) folder can be used to test the process specification for validity and consistent "style". It also allows rendering the processes in a web browser. Check the [development documentation](dev/README.md) for details.
 
 ## Process
 

and.json

@@ -1,7 +1,7 @@
 {
     "id": "and",
     "summary": "Logical AND",
-    "description": "Checks if **both** values are true.\n\nEvaluates parameter `x` before `y` and stops once the outcome is unambiguous. If any argument is `null`, the result will be `null` if the outcome is ambiguous.\n\n**Truth table:**\n\n```\na \\ b || null  | false | true\n----- || ----- | ----- | -----\nnull  || null  | false | null\nfalse || false | false | false\ntrue  || null  | false | true\n```",
+    "description": "Checks if **both** values are true.\n\nEvaluates parameter `x` before `y` and stops once the outcome is unambiguous. If any argument is `null`, the result will be `null` if the outcome is ambiguous.\n\n**Truth table:**\n\n```\nx \\ y || null  | false | true\n----- || ----- | ----- | -----\nnull  || null  | false | null\nfalse || false | false | false\ntrue  || null  | false | true\n```",
     "categories": [
         "logic"
     ],
@@ -90,4 +90,4 @@
             "result": true
         }
     }
-}
+}

array_append.json

@@ -25,15 +25,20 @@
         },
         {
             "name": "label",
-            "description": "If the given array is a labeled array, a new label for the new value should be given. If not given or `null`, the array index as string is used as the label. If in any case the label exists, a `LabelExists` exception is thrown.",
+            "description": "Provides a label for the new value. If not given or `null`, the natural next array index as number is used as the label. If in any case the label exists, a `LabelExists` exception is thrown.\n\nThis parameter only applies if the given array is a labeled array. If a non-null values is provided and the array is not labeled, an `ArrayNotLabeled` exception is thrown.",
             "optional": true,
             "default": null,
-            "schema": {
-                "type": [
-                    "string",
-                    "null"
-                ]
-            }
+            "schema": [
+                {
+                    "type": "number"
+                },
+                {
+                    "type": "string"
+                },
+                {
+                    "type": "null"
+                }
+            ]
         }
     ],
     "returns": {
@@ -48,6 +53,9 @@
     "exceptions": {
         "LabelExists": {
             "message": "An array element with the specified label already exists."
+        },
+        "ArrayNotLabeled": {
+            "message": "A label can't be provided as the given array is not labeled."
         }
     },
     "examples": [

between.json

@@ -8,7 +8,7 @@
     "parameters": [
         {
             "name": "x",
-            "description": "The value to check.",
+            "description": "The value to check.\n\nThe no-data value `null` is passed through and therefore gets propagated.",
             "schema": {
                 "description": "Any data type is allowed."
             }
@@ -38,7 +38,7 @@
         }
     ],
     "returns": {
-        "description": "`true` if `x` is between the specified bounds, otherwise `false`.",
+        "description": "`true` if `x` is between the specified bounds, `null` if `x` is a no-data value, `false` otherwise.",
         "schema": {
             "type": [
                 "boolean",

clip.json

@@ -1,7 +1,7 @@
 {
     "id": "clip",
     "summary": "Clip a value between a minimum and a maximum",
-    "description": "Clips a number between specified minimum and maximum values. A value larger than the maximum value is set to the maximum value, a value lower than the minimum value is set to the minimum value.\n\nThe no-data value `null` is passed through and therefore gets propagated.",
+    "description": "Clips a number between specified minimum and maximum values. A value larger than the maximum value is set to the maximum value, a value lower than the minimum value is set to the minimum value. If the maximum value is smaller than the minimum number, the process throws a `MinMaxSwapped` exception.\n\nThe no-data value `null` is passed through and therefore gets propagated.",
     "categories": [
         "math"
     ],
@@ -40,6 +40,11 @@
             ]
         }
     },
+    "exceptions": {
+        "MinMaxSwapped": {
+            "message": "The minimum value should be lower than or equal to the maximum value."
+        }
+    },
     "examples": [
         {
             "arguments": {
@@ -73,34 +78,5 @@
             },
             "returns": null
         }
-    ],
-    "process_graph": {
-        "min": {
-            "process_id": "min",
-            "arguments": {
-                "data": [
-                    {
-                        "from_parameter": "max"
-                    },
-                    {
-                        "from_parameter": "x"
-                    }
-                ]
-            }
-        },
-        "max": {
-            "process_id": "max",
-            "arguments": {
-                "data": [
-                    {
-                        "from_parameter": "min"
-                    },
-                    {
-                        "from_node": "min"
-                    }
-                ]
-            },
-            "result": true
-        }
-    }
-}
+    ]
+}

tests/.words → dev/.words

@@ -23,6 +23,7 @@ orthorectified
 radiometrically
 reflectances
 reproject
+reprojected
 Reprojects
 resample
 resampled
@@ -47,3 +48,10 @@ Hyndman
 date1
 date2
 favor
+XGBoost
+Chen
+Guestrin
+early_stopping_rounds
+Subsample
+hessian
+overfitting

eq.json

@@ -38,7 +38,8 @@
                 "type": [
                     "number",
                     "null"
-                ]
+                ],
+                "minimumExclusive": 0
             },
             "default": null,
             "optional": true

filter_bbox.json

@@ -1,7 +1,7 @@
 {
     "id": "filter_bbox",
     "summary": "Spatial filter using a bounding box",
-    "description": "Limits the data cube to the specified bounding box.\n\n* For raster data cubes, the filter retains a pixel in the data cube if the point at the pixel center intersects with the bounding box (as defined in the Simple Features standard by the OGC). Alternatively, ``filter_spatial()`` can be used to filter by geometry.\n* For vector data cubes, the filter retains the geometry in the data cube if the geometry is fully within the bounding box (as defined in the Simple Features standard by the OGC). All geometries that were empty or not contained fully within the bounding box will be removed from the data cube.\n\nAlternatively, ``filter_vector()`` can be used to filter by geometry.",
+    "description": "Limits the data cube to the specified bounding box.\n\n* For raster data cubes, the filter retains a pixel in the data cube if the point at the pixel center intersects with the bounding box (as defined in the Simple Features standard by the OGC). Alternatively, ``filter_spatial()`` can be used to filter by geometry.\n* For vector data cubes, the filter retains the geometry in the data cube if the geometry is fully within the bounding box (as defined in the Simple Features standard by the OGC). All geometries that were empty or not contained fully within the bounding box will be removed from the data cube.\n\nAlternatively, filter spatially with geometries using ``filter_spatial()`` (on a raster data cube) or ``filter_vector()`` (on a vector data cube).",
     "categories": [
         "cubes",
         "filter"
@@ -39,7 +39,7 @@
         },
         {
             "name": "extent",
-            "description": "A bounding box, which may include a vertical axis (see `base` and `height`).",
+            "description": "A bounding box, which may include a vertical axis (see `base` and `height`).\n\nIf the bounding box is not provided in the coordinate reference system (CRS) of the data cube, the bounding box is reprojected to the CRS of the spatial data cube dimensions.",
             "schema": {
                 "type": "object",
                 "subtype": "bounding-box",

filter_spatial.json

@@ -1,7 +1,7 @@
 {
     "id": "filter_spatial",
     "summary": "Spatial filter raster data cubes using geometries",
-    "description": "Limits the raster data cube over the spatial dimensions to the specified geometries.\n\n- For **polygons**, the filter retains a pixel in the data cube if the point at the pixel center intersects with at least one of the polygons (as defined in the Simple Features standard by the OGC).\n- For **points**, the process considers the closest pixel center.\n- For **lines** (line strings), the process considers all the pixels whose centers are closest to at least one point on the line.\n\nMore specifically, pixels outside of the bounding box of the given geometry will not be available after filtering. All pixels inside the bounding box that are not retained will be set to `null` (no data).\n\n Alternatively, use ``filter_bbox()`` to filter by bounding box.",
+    "description": "Limits the raster data cube over the spatial dimensions to the specified geometries.\n\n- For **polygons**, the filter retains a pixel in the data cube if the point at the pixel center intersects with at least one of the polygons (as defined in the Simple Features standard by the OGC).\n- For **points**, the process considers the closest pixel center.\n- For **lines** (line strings), the process considers all the pixels whose centers are closest to at least one point on the line.\n\nMore specifically, pixels outside of the bounding box of the given geometries will not be available after filtering. All pixels inside the bounding box that are not retained will be set to `null` (no data).\n\n Alternatively, use ``filter_bbox()`` to filter with a bounding box or ``filter_vector()`` to filter a vector data cube based on geometries. Use ``mask_polygon()`` to mask without changing the spatial extent of your data cube.",
     "categories": [
         "cubes",
         "filter"
@@ -26,7 +26,7 @@
         },
         {
             "name": "geometries",
-            "description": "One or more geometries used for filtering, given as GeoJSON or vector data cube. If multiple geometries are provided, the union of them is used. Empty geometries are ignored.\n\nLimits the data cube to the bounding box of the given geometries. No implicit masking gets applied. To mask the pixels of the data cube use ``mask_polygon()``.",
+            "description": "One or more geometries used for spatial filtering and masking, given as GeoJSON or vector data cube.",
             "schema": [
                 {
                     "title": "Vector Data Cube",

linear_scale_range.json

@@ -1,7 +1,7 @@
 {
     "id": "linear_scale_range",
     "summary": "Linear transformation between two ranges",
-    "description": "Performs a linear transformation between the input and output range.\n\nThe given number in `x` is clipped to the bounds specified in `inputMin` and `inputMax` so that the underlying formula *`((x - inputMin) / (inputMax - inputMin)) * (outputMax - outputMin) + outputMin`* never returns any value lower than `outputMin` or greater than `outputMax`.\n\nPotential use case include\n\n* scaling values to the 8-bit range (0 - 255) often used for numeric representation of values in one of the channels of the [RGB colour model](https://en.wikipedia.org/wiki/RGB_color_model#Numeric_representations) or\n* calculating percentages (0 - 100).\n\nThe no-data value `null` is passed through and therefore gets propagated.",
+    "description": "Performs a linear transformation between the input and output range.\n\nThe given number in `x` is clipped to the bounds specified in `inputMin` and `inputMax` so that the underlying formula *`((x - inputMin) / (inputMax - inputMin)) * (outputMax - outputMin) + outputMin`* never returns a value outside of the range defined by `outputMin` and `outputMax`.\n\nPotential use case include\n\n* scaling values to the 8-bit range (0 - 255) often used for numeric representation of values in one of the channels of the [RGB colour model](https://en.wikipedia.org/wiki/RGB_color_model#Numeric_representations) or\n* calculating percentages (0 - 100).\n\nThe no-data value `null` is passed through and therefore gets propagated.",
     "categories": [
         "math"
     ],
@@ -166,4 +166,4 @@
             "result": true
         }
     }
-}
+}

load_collection.json

@@ -64,7 +64,7 @@
                             "default": null
                         },
                         "crs": {
-                            "description": "Coordinate reference system of the extent, specified as as [EPSG code](http://www.epsg-registry.org/) or [WKT2 CRS string](http://docs.opengeospatial.org/is/18-010r7/18-010r7.html). Defaults to `4326` (EPSG code 4326) unless the client explicitly requests a different coordinate reference system.",
+                            "description": "Coordinate reference system of the extent, specified as as [EPSG code](http://www.epsg-registry.org/) or [WKT2 CRS string](http://docs.opengeospatial.org/is/18-010r7/18-010r7.html). Defaults to `4326` (EPSG code 4326) unless the client explicitly requests a different coordinate reference system. If the bounding box is not provided in the coordinate reference system (CRS) of the data cube, the bounding box is reprojected to the CRS of the spatial data cube dimensions.",
                             "anyOf": [
                                 {
                                     "title": "EPSG Code",

meta/subtype-schemas.json

@@ -232,6 +232,12 @@
                 }
             }
         },
+        "ml-model": {
+            "type": "object",
+            "subtype": "ml-model",
+            "title": "Machine Learning Model",
+            "description": "A machine learning model, accompanied with STAC metadata that implements the the STAC ml-model extension."
+        },
         "output-format": {
             "type": "string",
             "subtype": "output-format",
@@ -420,4 +426,4 @@
             "description": "Year as integer, can be any number of digits and can be negative."
         }
     }
-}
+}

neq.json

@@ -38,7 +38,8 @@
                 "type": [
                     "number",
                     "null"
-                ]
+                ],
+                "minimumExclusive": 0
             },
             "default": null,
             "optional": true

or.json

@@ -1,7 +1,7 @@
 {
     "id": "or",
     "summary": "Logical OR",
-    "description": "Checks if **at least one** of the values is true. Evaluates parameter `x` before `y` and stops once the outcome is unambiguous. If a component is `null`, the result will be `null` if the outcome is ambiguous.\n\n**Truth table:**\n\n```\na \\ b || null | false | true\n----- || ---- | ----- | ----\nnull  || null | null  | true\nfalse || null | false | true\ntrue  || true | true  | true\n```",
+    "description": "Checks if **at least one** of the values is true. Evaluates parameter `x` before `y` and stops once the outcome is unambiguous. If a component is `null`, the result will be `null` if the outcome is ambiguous.\n\n**Truth table:**\n\n```\nx \\ y || null | false | true\n----- || ---- | ----- | ----\nnull  || null | null  | true\nfalse || null | false | true\ntrue  || true | true  | true\n```",
     "categories": [
         "logic"
     ],
@@ -90,4 +90,4 @@
             "result": true
         }
     }
-}
+}

proposals/filter_vector.json

@@ -1,7 +1,7 @@
 {
     "id": "filter_vector",
     "summary": "Spatial vector filter using geometries",
-    "description": "Limits the vector data cube to the specified geometries. The process works on geometries as defined in the Simple Features standard by the OGC. All geometries that were empty or become empty will be removed from the data cube. Alternatively, use ``filter_bbox()`` to filter by bounding box.",
+    "description": "Limits the vector data cube to the specified geometries. The process works on geometries as defined in the Simple Features standard by the OGC. All geometries that were empty or become empty will be removed from the data cube. Alternatively, use ``filter_bbox()`` to filter with a bounding box or ``filter_spatial()`` to filter a raster data cube based on geometries.",
     "categories": [
         "cubes",
         "filter",