Feature/multi asset StorageScheduler #1313

Tammevesky · 2025-01-21T13:54:56Z

Description

New service to create_sequential_scheduling_job
New service to create_simultaneous_scheduling_job
Test new services
Schema used by StorageScheduler accepts multi-asset flex-models
StorageScheduler transforms multi-asset flex-models into device_scheduler input

Look & Feel

This section can contain example pictures for UI, Input/Output for CLI, Request / Response for API endpoint, etc.

How to test

test_ceate_sequential_scheduling_job
test_ceate_simultaneous_scheduling_job
flexmeasures-client is not yet compatible, because the API is not yet compatible. This PR is only for (plugin) developers.

Further Improvements

In scope:

Added changelog item in documentation/changelog.rst under 'Infrastructure / Support' section, targeting (plugin) developers in this particular PR.

Follow-up issue:

API support for multi-asset flex-model (requires new endpoint): API endpoint for scheduling asset #1065

Related Items

Mention if this PR closes an Issue or Project.

Signed-off-by: F.N. Claessen <[email protected]>

…a scheduling job Signed-off-by: F.N. Claessen <[email protected]>

Signed-off-by: F.N. Claessen <[email protected]>

…ntext Signed-off-by: F.N. Claessen <[email protected]>

Signed-off-by: F.N. Claessen <[email protected]>

Signed-off-by: Victor Garcia Reolid <[email protected]>

…ing-asset' into feature/api/endpoint-for-scheduling-asset

This reverts commit ff22fea.

This reverts commit 4621ef6.

…ext" This reverts commit 8374a9c.

Signed-off-by: F.N. Claessen <[email protected]>

…alse Signed-off-by: F.N. Claessen <[email protected]>

Signed-off-by: F.N. Claessen <[email protected]>

Flix6x

Self-review, incl. 1 suggestion for @Tammevesky to take up.

flexmeasures/data/schemas/generic_assets.py

Flix6x · 2025-01-30T20:27:06Z

flexmeasures/data/schemas/scheduling/__init__.py

@@ -211,3 +219,91 @@ def _get_variable_quantity_unit(
                f"Unexpected type '{type(variable_quantity)}' for '{field}': {variable_quantity}."
            )
        return unit
+
+
+class SequentialFlexModelSchema(Schema):


Currently being used for both simultaneous and sequential multi-asset scheduling, but not meant to be used for the simultaneous case. Misusing it for that is likely causing the double nesting of the sensor-flex-model field.

Flix6x · 2025-01-30T20:28:13Z

flexmeasures/data/schemas/scheduling/__init__.py

+        return {"sensor-flex-model": extra, **rest}
+
+
+class AssetTriggerSchema(Schema):


Cherry-picked, but currently not used in this PR. May belong in #1065 instead.

flexmeasures/data/schemas/sensors.py

Flix6x · 2025-01-30T20:33:25Z

flexmeasures/data/services/scheduling.py

+) -> list[Job]:
+    jobs = []
+
+    # scheduler_kwargs["resolution"] = sensor.event_resolution  # todo: needed?


Adding a test involving two flexible devices with different resolutions should clear this up.

Flix6x · 2025-01-30T20:37:48Z

flexmeasures/data/services/scheduling.py

+    force_new_job_creation: bool = False,
+    scheduler_specs: dict | None = None,
+    depends_on: list[Job] | None = None,
+    **scheduler_kwargs,


scheduler_kwargs is in desperate need of a docstring example, especially for its flex_model after we'll (partially?) deserialize it by passing it through a to-be-created SimultaneousFlexModelSchema.

Flix6x · 2025-01-30T20:37:59Z

flexmeasures/data/services/scheduling.py

+    force_new_job_creation: bool = False,
+    scheduler_specs: dict | None = None,
+    depends_on: list[Job] | None = None,
+    **scheduler_kwargs,


scheduler_kwargs is in desperate need of a docstring example, especially for its flex_model, which is partially deserialized after passing through SequentialFlexModelSchema.

Flix6x · 2025-01-30T20:42:58Z

flexmeasures/data/tests/test_scheduling_simultaneous.py

+    assert all(ev_power.loc[start_charging:end_charging] == -0.01)  # 10 kW
+    assert all(battery_power.loc[start_charging:end_charging] == 0.01)  # 10 kW


@svenay We should adjust this test so our expectations here differ from that of the sequential case.

Flix6x · 2025-01-30T20:47:59Z

flexmeasures/data/models/planning/storage.py


        if soc_targets and not isinstance(soc_targets, Sensor):
            max_target_datetime = max([soc_target["end"] for soc_target in soc_targets])
            if max_target_datetime > self.end:
-                max_server_horizon = get_max_planning_horizon(self.resolution)


My first thought is that this function still does what it is supposed to do, namely extending the planning horizon if needed, therein taking into account the resolution of each device's power sensor. But it deserves a closer look, especially regarding how the horizon is derived using the resolution.

flexmeasures/data/models/planning/storage.py

…straints for device d rather than device 0. Signed-off-by: F.N. Claessen <[email protected]>

Signed-off-by: F.N. Claessen <[email protected]>

…multi-asset-StorageScheduler

Signed-off-by: F.N. Claessen <[email protected]>

…multi-asset-StorageScheduler

Signed-off-by: F.N. Claessen <[email protected]>

…ases-and-breaking-changes-for-github-actions/#actions-cache-v1-v2-and-actions-toolkit-cache-package-closing-down Signed-off-by: F.N. Claessen <[email protected]>

Signed-off-by: F.N. Claessen <[email protected]>

…a future version, please use 'h' instead. Signed-off-by: F.N. Claessen <[email protected]>

…fields is deprecated. Use 'dump_default' instead. Signed-off-by: F.N. Claessen <[email protected]>

…ated with new scheduler data source "<Data source 12 (None's SuccessfulScheduler model vNone)>".` Signed-off-by: F.N. Claessen <[email protected]>

Signed-off-by: F.N. Claessen <[email protected]>

This reverts commit bad7cba.

Signed-off-by: F.N. Claessen <[email protected]>

Flix6x and others added 30 commits May 16, 2024 16:56

feature: copy trigger_schedule endpoint from SensorAPI to AssetAPI

e2fced5

Signed-off-by: F.N. Claessen <[email protected]>

fix: update function signature

bfdd94f

Signed-off-by: F.N. Claessen <[email protected]>

fix: get rid of deprecation warning

5795e1b

Signed-off-by: F.N. Claessen <[email protected]>

fix: update docstring

2e4cfe0

Signed-off-by: F.N. Claessen <[email protected]>

fix: imports

2e28ed9

Signed-off-by: F.N. Claessen <[email protected]>

fix: obtain sensors from flex-model

f3711de

Signed-off-by: F.N. Claessen <[email protected]>

fix: create one scheduling job for each sensor listed in the flex-model

305d76d

Signed-off-by: F.N. Claessen <[email protected]>

fix: add todos

f6e8075

Signed-off-by: F.N. Claessen <[email protected]>

docs: update endpoint main descriptions and quickrefs

4bf91dc

Signed-off-by: F.N. Claessen <[email protected]>

style: black

0917896

Signed-off-by: F.N. Claessen <[email protected]>

fix: changelog syntax

753b411

Signed-off-by: F.N. Claessen <[email protected]>

fix: remove redundant session commit when calling the API to trigger …

ba90dd1

…a scheduling job Signed-off-by: F.N. Claessen <[email protected]>

docs: update FlexContextSchema docstring

af4debf

Signed-off-by: F.N. Claessen <[email protected]>

feature: helpful message for test developers

49a1a12

Signed-off-by: F.N. Claessen <[email protected]>

feature: check auth on sensors referenced in flex-context

8a3e112

Signed-off-by: F.N. Claessen <[email protected]>

feature: allow checking permissions on optional fields

25d1180

Signed-off-by: F.N. Claessen <[email protected]>

feature: decorator supports custom error handler

21ac09a

Signed-off-by: F.N. Claessen <[email protected]>

docs: add inline note explaining status code

b4ad01a

Signed-off-by: F.N. Claessen <[email protected]>

feature: flex_context_loader lists all sensors contained in a flex-co…

5b4c30f

…ntext Signed-off-by: F.N. Claessen <[email protected]>

feature: support context loader that returns multiple contexts

484a022

Signed-off-by: F.N. Claessen <[email protected]>

feature: check permissions on sensors referenced in flex-context

8374a9c

Signed-off-by: F.N. Claessen <[email protected]>

feature: add test checking permissions

4621ef6

Signed-off-by: F.N. Claessen <[email protected]>

fix: response with field names

ff22fea

Signed-off-by: F.N. Claessen <[email protected]>

add create_sequential_scheduling_job function

be51f1b

Signed-off-by: Victor Garcia Reolid <[email protected]>

add fixtures

061e1f3

Signed-off-by: Victor Garcia Reolid <[email protected]>

add test_create_sequential_jobs

6eb122b

Signed-off-by: Victor Garcia Reolid <[email protected]>

Merge remote-tracking branch 'origin/feature/api/endpoint-for-schedul…

b204e4b

…ing-asset' into feature/api/endpoint-for-scheduling-asset

Revert "fix: response with field names"

22834ca

This reverts commit ff22fea.

Revert "feature: add test checking permissions"

dc07b11

This reverts commit 4621ef6.

Revert "feature: check permissions on sensors referenced in flex-cont…

b583c89

…ext" This reverts commit 8374a9c.

Flix6x added 8 commits January 30, 2025 16:39

fix: use the right sensor resolution

c767e61

Signed-off-by: F.N. Claessen <[email protected]>

fix: return schedules for all storage devices

7d2f86c

Signed-off-by: F.N. Claessen <[email protected]>

fix: return schedule for a single storage device if return_multiple=F…

cf63eb1

…alse Signed-off-by: F.N. Claessen <[email protected]>

docs: fix docstring

22c7bc0

Signed-off-by: F.N. Claessen <[email protected]>

fix: new argument is optional

eccd175

Signed-off-by: F.N. Claessen <[email protected]>

fix: StorageFallbackScheduler

d621b35

Signed-off-by: F.N. Claessen <[email protected]>

fix: update tests

f1954a1

Signed-off-by: F.N. Claessen <[email protected]>

fix: sensor attribute is called event_resolution

5648956

Signed-off-by: F.N. Claessen <[email protected]>

Flix6x requested changes Jan 30, 2025

View reviewed changes

Flix6x added 21 commits January 31, 2025 09:57

fix: The fallback schedule for device d should be based on device con…

5349dbb

…straints for device d rather than device 0. Signed-off-by: F.N. Claessen <[email protected]>

revert: changes to data/schemas/generic_assets.py

ac653c2

Signed-off-by: F.N. Claessen <[email protected]>

revert: changes to data/schemas/sensors.py

5834faa

Signed-off-by: F.N. Claessen <[email protected]>

Merge remote-tracking branch 'refs/remotes/origin/main' into feature/…

520834f

…multi-asset-StorageScheduler

dev: xfail

2826164

Signed-off-by: F.N. Claessen <[email protected]>

Merge remote-tracking branch 'refs/remotes/origin/main' into feature/…

b57b04f

…multi-asset-StorageScheduler

fix: prefer charging sooner

c564bdd

Signed-off-by: F.N. Claessen <[email protected]>

dev: https://github.blog/changelog/2024-12-05-notice-of-upcoming-rele…

bd941c9

…ases-and-breaking-changes-for-github-actions/#actions-cache-v1-v2-and-actions-toolkit-cache-package-closing-down Signed-off-by: F.N. Claessen <[email protected]>

dev: update actions/cache version

0803573

Signed-off-by: F.N. Claessen <[email protected]>

refactor: simplify preference check

ef52f5c

Signed-off-by: F.N. Claessen <[email protected]>

fix: more explicit default preference to charge sooner

af999bf

Signed-off-by: F.N. Claessen <[email protected]>

fix: not yet enqueued jobs might not have a status yet

3c0debb

Signed-off-by: F.N. Claessen <[email protected]>

fix: resolve FutureWarning: 'H' is deprecated and will be removed in …

3866f5e

…a future version, please use 'h' instead. Signed-off-by: F.N. Claessen <[email protected]>

fix: resolve RemovedInMarshmallow4Warning: The 'default' argument to …

3daf07c

…fields is deprecated. Use 'dump_default' instead. Signed-off-by: F.N. Claessen <[email protected]>

feat: improve test logging, avoiding log statements like `Session upd…

b28a2e4

…ated with new scheduler data source "<Data source 12 (None's SuccessfulScheduler model vNone)>".` Signed-off-by: F.N. Claessen <[email protected]>

docs: rephrase inline comment

8cdf117

Signed-off-by: F.N. Claessen <[email protected]>

dev: pin rq

bad7cba

Signed-off-by: F.N. Claessen <[email protected]>

Revert "dev: pin rq"

5f54726

This reverts commit bad7cba.

fix: pass connection (needed with rq=2)

baaa461

Signed-off-by: F.N. Claessen <[email protected]>

feat: show traceback in failing test

d2369c3

Signed-off-by: F.N. Claessen <[email protected]>

fix: Python 3.8 compatibility

794e1c2

Signed-off-by: F.N. Claessen <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/multi asset StorageScheduler #1313

Feature/multi asset StorageScheduler #1313

Tammevesky commented Jan 21, 2025 •

edited by Flix6x

Loading

Flix6x left a comment

Flix6x Jan 30, 2025

Flix6x Jan 30, 2025

Flix6x Jan 30, 2025

Flix6x Jan 30, 2025

Flix6x Jan 30, 2025

Flix6x Jan 30, 2025

Flix6x Jan 30, 2025

		return {"sensor-flex-model": extra, **rest}


		class AssetTriggerSchema(Schema):

		assert all(ev_power.loc[start_charging:end_charging] == -0.01) # 10 kW
		assert all(battery_power.loc[start_charging:end_charging] == 0.01) # 10 kW

Feature/multi asset StorageScheduler #1313

Are you sure you want to change the base?

Feature/multi asset StorageScheduler #1313

Conversation

Tammevesky commented Jan 21, 2025 • edited by Flix6x Loading

Description

Look & Feel

How to test

Further Improvements

Related Items

Flix6x left a comment

Choose a reason for hiding this comment

Flix6x Jan 30, 2025

Choose a reason for hiding this comment

Flix6x Jan 30, 2025

Choose a reason for hiding this comment

Flix6x Jan 30, 2025

Choose a reason for hiding this comment

Flix6x Jan 30, 2025

Choose a reason for hiding this comment

Flix6x Jan 30, 2025

Choose a reason for hiding this comment

Flix6x Jan 30, 2025

Choose a reason for hiding this comment

Flix6x Jan 30, 2025

Choose a reason for hiding this comment

Tammevesky commented Jan 21, 2025 •

edited by Flix6x

Loading