Skip to content

Commit

Permalink
Merge branch 'geopython:master' into numbermatched_merge
Browse files Browse the repository at this point in the history
  • Loading branch information
Moritz-Langer authored Jan 22, 2025
2 parents 9c5a5cb + 29b507f commit 69712bb
Show file tree
Hide file tree
Showing 24 changed files with 281 additions and 1,437 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
- name: Install and run OpenSearch 📦
uses: esmarkowski/[email protected]
with:
version: 2.12.0
version: 2.18.0
security-disabled: true
port: 9209
- name: Install and run MongoDB
Expand Down
52 changes: 29 additions & 23 deletions docs/source/cql.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,18 @@
CQL support
===========

OGC Common Query Language (`CQL2`_) is a generic language designed to provide enhanced query and subset/filtering to (primarily) feature and record data.

Providers
---------

As of now the available providers supported for CQL filtering are limited to :ref:`Elasticsearch <Elasticsearch>` and :ref:`PostgreSQL <PostgreSQL>`.

CQL2 support is implemented in various pygeoapi feature and record providers. See the :ref:`feature <ogcapi-features>` and :ref:`metadata <ogcapi-records>` provider sections
for current provider support.

Limitations
-----------

Support of CQL is limited to `Simple CQL filter <https://portal.ogc.org/files/96288#cql-core>`_ and thus it allows to query with the
Support of CQL is limited to `Basic CQL2 <https://docs.ogc.org/is/21-065r2/21-065r2.html#cql2-core>`_ and thus it allows to query with the
following predicates:

* comparison predicates
Expand All @@ -21,20 +24,20 @@ following predicates:
Formats
-------

At the moment Elasticsearch supports only the CQL dialect with the JSON encoding `CQL-JSON <https://portal.ogc.org/files/96288#simple-cql-JSON>`_.
Supported providers leverage the CQL2 dialect with the JSON encoding `CQL-JSON <https://docs.ogc.org/is/21-065r2/21-065r2.html#cql2-json>`_.

PostgreSQL supports both CQL-JSON and CQL-text dialects, `CQL-JSON <https://portal.ogc.org/files/96288#simple-cql-JSON>`_ and `CQL-TEXT <https://portal.ogc.org/files/96288#simple-cql-text>`_
PostgreSQL supports both `CQL2 JSON <https://docs.ogc.org/is/21-065r2/21-065r2.html#cql2-json>`_ and `CQL text <https://docs.ogc.org/is/21-065r2/21-065r2.html#cql2-text>`_ dialects.

Queries
^^^^^^^

The PostgreSQL provider uses `pygeofilter <https://github.com/geopython/pygeofilter>`_ allowing a range of filter expressions, see examples for:

* `Comparison predicates <https://portal.ogc.org/files/96288#simple-cql_comparison-predicates>`_
* `Spatial predicates <https://portal.ogc.org/files/96288#enhanced-spatial-operators>`_
* `Temporal predicates <https://portal.ogc.org/files/96288#simple-cql_temporal>`_
* `Comparison predicates (`Advanced <https://docs.ogc.org/is/21-065r2/21-065r2.html#advanced-comparison-operators>`_, `Case-insensitive <https://docs.ogc.org/is/21-065r2/21-065r2.html#case-insensitive-comparison>`_)
* `Spatial predicates <https://docs.ogc.org/is/21-065r2/21-065r2.html#spatial-functions>`_
* `Temporal predicates <https://docs.ogc.org/is/21-065r2/21-065r2.html#temporal-functions>`_

Using Elasticsearch the following type of queries are supported right now:
Using Elasticsearch the following type of queries are supported currently:

* ``between`` predicate query
* Logical ``and`` query with ``between`` and ``eq`` expression
Expand All @@ -59,11 +62,11 @@ A ``BETWEEN`` example for a specific property through an HTTP POST request:
curl --location --request POST 'http://localhost:5000/collections/nhsl_hazard_threat_all_indicators_s_bc/items?f=json&limit=50&filter-lang=cql-json' \
--header 'Content-Type: application/query-cql-json' \
--data-raw '{
"between": {
"value": { "property": "properties.MHn_Intensity" },
"lower": 0.59,
"upper": 0.60
}
"op": "between",
"args": [
{"property": "properties.MHn_Intensity"},
[0.59, 0.60]
]
}'
Or
Expand All @@ -73,11 +76,11 @@ Or
curl --location --request POST 'http://localhost:5000/collections/recentearthquakes/items?f=json&limit=10&filter-lang=cql-json'
--header 'Content-Type: application/query-cql-json'
--data-raw '{
"between":{
"value":{"property": "ml"},
"lower":4,
"upper":4.5
}
"op": "between",
"args": [
{"property": "ml"},
[4, 4.5]
]
}'
The same ``BETWEEN`` query using HTTP GET request formatted as CQL text and URL encoded as below:
Expand All @@ -93,7 +96,11 @@ An ``EQUALS`` example for a specific property:
curl --location --request POST 'http://localhost:5000/collections/recentearthquakes/items?f=json&limit=10&filter-lang=cql-json'
--header 'Content-Type: application/query-cql-json'
--data-raw '{
"eq":[{"property": "user_entered"},"APBE"]
"op": "=",
"args": [
{"property": "user_entered"},
"APBE"
]
}'
A ``CROSSES`` example via an HTTP GET request. The CQL text is passed via the ``filter`` parameter.
Expand All @@ -115,7 +122,6 @@ The same example, but this time providing a geometry in EWKT format:
curl "http://localhost:5000/collections/beni/items?filter=DWITHIN(geometry,SRID=3857;POINT(1392921%205145517),100,meters)"
Note that the CQL text has been URL encoded. This is required in curl commands but when entering in a browser, plain text can be used e.g. ``CROSSES(foo_geom, LINESTRING(28 -2, 30 -4))``.

.. _`CQL2`: https://docs.ogc.org/is/21-065r2/21-065r2.html
38 changes: 0 additions & 38 deletions docs/source/development.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,44 +22,6 @@ Tests can be run locally as part of development workflow. They are also run on
To run all tests, simply run ``pytest`` in the repository. To run a specific test file,
run ``pytest tests/api/test_itemtypes.py``, for example.


CQL extension lifecycle
-----------------------

Limitations
^^^^^^^^^^^

This workflow is valid only for the `CQL-JSON` format.

Schema
^^^^^^

The Common Query Language (CQL) is the part 3 of the standard OGC API - Features. This extension has its specification available at
`OGC API - Features - Part 3: Filtering and the Common Query Language (CQL) <https://portal.ogc.org/files/96288>`_ and the schema exists in development at
`cql.json <https://portal.ogc.org/files/96288#cql-json-schema>`_.

Model generation
^^^^^^^^^^^^^^^^

pygeoapi uses a class-based Python model interface to translate the schema into Python objects defined by `pydantic <https://docs.pydantic.dev/>`_ models.
The model is generated with the pre-processing of the schema through the utility ``datamodel-codegen``, which is part
of the `datamodel-code-generator <https://koxudaxi.github.io/datamodel-code-generator/>`_ package:


.. code-block:: bash
# Generate from local downloaded json schema file
datamodel-codegen --input ~/Download/cql-schema.json --input-file-type jsonschema --output ./pygeoapi/models/cql_update.py --class-name CQLModel
Note that datamodel-code-generator must be explicitly installed, as it is not a pygeoapi runtime dependency

How to merge
^^^^^^^^^^^^

Once the new pydantic models have been generated then the content of the Python file ``cql_update.py`` can be used to replace the old classes within the ``cql.py`` file.
Update everything above the function ``get_next_node`` and then verify if the tests for the CQL are still passing, for example ``test_post_cql_json_between_query``
in ``tests/test_elasticsearch__provider.py``.

Working with Spatialite on OSX
------------------------------

Expand Down
10 changes: 10 additions & 0 deletions pygeoapi/api/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -1161,6 +1161,16 @@ def describe_collections(api: API, request: APIRequest,
'cellsCount': p._coverage_properties['height'],
'resolution': p._coverage_properties['resy']
}]
if 'time_range' in p._coverage_properties:
collection['extent']['temporal'] = {
'interval': [p._coverage_properties['time_range']]
}
if 'restime' in p._coverage_properties:
collection['extent']['temporal']['grid'] = {
'resolution': p._coverage_properties['restime'] # noqa
}
if 'uad' in p._coverage_properties:
collection['extent'].update(p._coverage_properties['uad']) # noqa

try:
tile = get_provider_by_type(v['providers'], 'tile')
Expand Down
Loading

0 comments on commit 69712bb

Please sign in to comment.