Skip to content

Commit

Permalink
Data prefetch ems for feature (#420)
Browse files Browse the repository at this point in the history 
* Pre-fetch data

* maintanance and extend tests

* comment clean up

* nansum usage (to be save)

* Feature/config nested (#421)

* Nested config, devices registry

 * All config now nested.
    - Use default config from model field default values. If providers
      should be enabled by default, non-empty default config file could
      be provided again.
    - Environment variable support with EOS_ prefix and __ between levels,
      e.g. EOS_SERVER__EOS_SERVER_PORT=8503 where all values are case
      insensitive.
      For more information see:
      https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values
    - Use devices as registry for configured devices. DeviceBase as base
      class with for now just initializion support (in the future expand
      to operations during optimization).
    - Strip down ConfigEOS to the only configuration instance. Reload
      from file or reset to defaults is possible.

 * Fix multi-initialization of derived SingletonMixin classes.

* Documentation: Support nested config

 * Add examples to pydantic models.

* EOSdash: Support nested types

* Rename settings variables (remove prefixes)

* Fix API endpoint

* Fix EOSdash startup (docker)

 * Docker: Copy the same directory structure (src/) to support the
   lifespan startup of EOSdash.
   Use EOS_SERVER_EOSDASH_SESSKEY environment variable to provide
   EOSdash with session key.

* PR review

* PVForecast: planes as nested config (list)

* Update manual documentation for nested config.

 * Add config_file_path, config_folder_path back to general
   (ConfigCommonSettings). Overwrite in docs generation.

* Config: Move lat/long/timezone from prediction to general

* Docs: Add global example documentation.

 * merge_models: Use deecopy to not change input data.

* EOSdash: Sort config by name

* Review comments

* Feature/config nested dependabot req. (#415)

* Bump numpydantic from 1.6.4 to 1.6.7 (#413)

Bumps [numpydantic](https://github.com/p2p-ld/numpydantic) from 1.6.4 to 1.6.7.
- [Release notes](https://github.com/p2p-ld/numpydantic/releases)
- [Changelog](https://github.com/p2p-ld/numpydantic/blob/main/docs/changelog.md)
- [Commits](p2p-ld/numpydantic@v1.6.4...v1.6.7)

---
updated-dependencies:
- dependency-name: numpydantic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump timezonefinder from 6.5.7 to 6.5.8 (#414)

Bumps [timezonefinder](https://github.com/jannikmi/timezonefinder) from 6.5.7 to 6.5.8.
- [Release notes](https://github.com/jannikmi/timezonefinder/releases)
- [Changelog](https://github.com/jannikmi/timezonefinder/blob/master/CHANGELOG.rst)
- [Commits](jannikmi/timezonefinder@6.5.7...6.5.8)

---
updated-dependencies:
- dependency-name: timezonefinder
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pydantic from 2.10.5 to 2.10.6 (#412)

Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.10.5 to 2.10.6.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](pydantic/pydantic@v2.10.5...v2.10.6)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump fastapi[standard] from 0.115.6 to 0.115.7 (#411)

Bumps [fastapi[standard]](https://github.com/fastapi/fastapi) from 0.115.6 to 0.115.7.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](fastapi/fastapi@0.115.6...0.115.7)

---
updated-dependencies:
- dependency-name: fastapi[standard]
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Readme: Add hint for interfering ports on Synology Closes #408 (#419)

* Pics or it didn't happen (#402)

* inverter added

* png creation

* save svg into cache folder

* mypy

* comment

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Dominique Lasserre <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* inverter, prediction.hours

* self.config.general.data_cache_path

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Dominique Lasserre <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
  • Loading branch information
@NormannK @Lasall @dependabot
3 people authored Jan 26, 2025
1 parent 90688a3 commit 480adf8
Show file tree
Hide file tree
Showing 4 changed files with 175 additions and 116 deletions.
6 changes: 0 additions & 6 deletions single_test_optimization.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -164,9 +164,6 @@ def prepare_optimization_real_parameters() -> OptimizationParameters:
"max_charge_power_w": 11040,
"initial_soc_percentage": 5,
},
"inverter": {
"max_power_wh": 10000,
},
"temperature_forecast": temperature_forecast,
"start_solution": start_solution,
}
Expand Down Expand Up @@ -321,9 +318,6 @@ def prepare_optimization_parameters() -> OptimizationParameters:
"max_charge_power_w": 11040,
"initial_soc_percentage": 5,
},
"inverter": {
"max_power_wh": 10000,
},
"temperature_forecast": temperature_forecast,
"start_solution": start_solution,
}
Expand Down
205 changes: 98 additions & 107 deletions src/akkudoktoreos/core/ems.py
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from typing import Any, ClassVar, Dict, Optional, Union
from typing import Any, ClassVar, Optional

import numpy as np
from numpydantic import NDArray, Shape
Expand Down Expand Up @@ -191,7 +191,7 @@ def set_parameters(
len(self.load_energy_array), parameters.einspeiseverguetung_euro_pro_wh, float
)
)
if inverter is not None:
if inverter:
self.battery = inverter.battery
else:
self.battery = None
Expand All @@ -203,7 +203,7 @@ def set_parameters(
self.ev_charge_hours = np.full(self.config.prediction.hours, 0.0)

def set_akku_discharge_hours(self, ds: np.ndarray) -> None:
if self.battery is not None:
if self.battery:
self.battery.set_discharge_per_hour(ds)

def set_akku_ac_charge_hours(self, ds: np.ndarray) -> None:
Expand All @@ -216,7 +216,7 @@ def set_ev_charge_hours(self, ds: np.ndarray) -> None:
self.ev_charge_hours = ds

def set_home_appliance_start(self, ds: int, global_start_hour: int = 0) -> None:
if self.home_appliance is not None:
if self.home_appliance:
self.home_appliance.set_starting_time(ds, global_start_hour=global_start_hour)

def reset(self) -> None:
Expand Down Expand Up @@ -281,53 +281,50 @@ def simulate_start_now(self) -> dict[str, Any]:
return self.simulate(start_hour)

def simulate(self, start_hour: int) -> dict[str, Any]:
"""hour.
"""Simulate energy usage and costs for the given start hour.
akku_soc_pro_stunde begin of the hour, initial hour state!
last_wh_pro_stunde integral of last hour (end state)
last_wh_pro_stunde integral of last hour (end state)
"""
# Check for simulation integrity
missing_data = []

if self.load_energy_array is None:
missing_data.append("Load Curve")
if self.pv_prediction_wh is None:
missing_data.append("PV Forecast")
if self.elect_price_hourly is None:
missing_data.append("Electricity Price")
if self.ev_charge_hours is None:
missing_data.append("EV Charge Hours")
if self.ac_charge_hours is None:
missing_data.append("AC Charge Hours")
if self.dc_charge_hours is None:
missing_data.append("DC Charge Hours")
if self.elect_revenue_per_hour_arr is None:
missing_data.append("Feed-in Tariff")
required_attrs = [
"load_energy_array",
"pv_prediction_wh",
"elect_price_hourly",
"ev_charge_hours",
"ac_charge_hours",
"dc_charge_hours",
"elect_revenue_per_hour_arr",
]
missing_data = [
attr.replace("_", " ").title() for attr in required_attrs if getattr(self, attr) is None
]

if missing_data:
error_msg = "Mandatory data missing - " + ", ".join(missing_data)
logger.error(error_msg)
raise ValueError(error_msg)
else:
# make mypy happy
assert self.load_energy_array is not None
assert self.pv_prediction_wh is not None
assert self.elect_price_hourly is not None
assert self.ev_charge_hours is not None
assert self.ac_charge_hours is not None
assert self.dc_charge_hours is not None
assert self.elect_revenue_per_hour_arr is not None

load_energy_array = self.load_energy_array

if not (
len(load_energy_array) == len(self.pv_prediction_wh) == len(self.elect_price_hourly)
):
error_msg = f"Array sizes do not match: Load Curve = {len(load_energy_array)}, PV Forecast = {len(self.pv_prediction_wh)}, Electricity Price = {len(self.elect_price_hourly)}"
logger.error("Mandatory data missing - %s", ", ".join(missing_data))
raise ValueError(f"Mandatory data missing: {', '.join(missing_data)}")

# Pre-fetch data
load_energy_array = np.array(self.load_energy_array)
pv_prediction_wh = np.array(self.pv_prediction_wh)
elect_price_hourly = np.array(self.elect_price_hourly)
ev_charge_hours = np.array(self.ev_charge_hours)
ac_charge_hours = np.array(self.ac_charge_hours)
dc_charge_hours = np.array(self.dc_charge_hours)
elect_revenue_per_hour_arr = np.array(self.elect_revenue_per_hour_arr)

# Fetch objects
battery = self.battery
assert battery # to please mypy
ev = self.ev
home_appliance = self.home_appliance
inverter = self.inverter

if not (len(load_energy_array) == len(pv_prediction_wh) == len(elect_price_hourly)):
error_msg = f"Array sizes do not match: Load Curve = {len(load_energy_array)}, PV Forecast = {len(pv_prediction_wh)}, Electricity Price = {len(elect_price_hourly)}"
logger.error(error_msg)
raise ValueError(error_msg)

# Optimized total hours calculation
end_hour = len(load_energy_array)
total_hours = end_hour - start_hour

Expand All @@ -337,116 +334,110 @@ def simulate(self, start_hour: int) -> dict[str, Any]:
consumption_energy_per_hour = np.full((total_hours), np.nan)
costs_per_hour = np.full((total_hours), np.nan)
revenue_per_hour = np.full((total_hours), np.nan)
soc_per_hour = np.full((total_hours), np.nan) # Hour End State
soc_per_hour = np.full((total_hours), np.nan)
soc_ev_per_hour = np.full((total_hours), np.nan)
losses_wh_per_hour = np.full((total_hours), np.nan)
home_appliance_wh_per_hour = np.full((total_hours), np.nan)
electricity_price_per_hour = np.full((total_hours), np.nan)

# Set initial state
if self.battery:
soc_per_hour[0] = self.battery.current_soc_percentage()
if self.ev:
soc_ev_per_hour[0] = self.ev.current_soc_percentage()
soc_per_hour[0] = battery.current_soc_percentage()
if ev:
soc_ev_per_hour[0] = ev.current_soc_percentage()

for hour in range(start_hour, end_hour):
hour_since_now = hour - start_hour
hour_idx = hour - start_hour

# save begin states
if self.battery:
soc_per_hour[hour_since_now] = self.battery.current_soc_percentage()
else:
soc_per_hour[hour_since_now] = 0.0
if self.ev:
soc_ev_per_hour[hour_since_now] = self.ev.current_soc_percentage()
soc_per_hour[hour_idx] = battery.current_soc_percentage()

if ev:
soc_ev_per_hour[hour_idx] = ev.current_soc_percentage()

# Accumulate loads and PV generation
consumption = self.load_energy_array[hour]
losses_wh_per_hour[hour_since_now] = 0.0
consumption = load_energy_array[hour]
losses_wh_per_hour[hour_idx] = 0.0

# Home appliances
if self.home_appliance:
ha_load = self.home_appliance.get_load_for_hour(hour)
if home_appliance:
ha_load = home_appliance.get_load_for_hour(hour)
consumption += ha_load
home_appliance_wh_per_hour[hour_since_now] = ha_load
home_appliance_wh_per_hour[hour_idx] = ha_load

# E-Auto handling
if self.ev:
if self.ev_charge_hours[hour] > 0:
loaded_energy_ev, verluste_eauto = self.ev.charge_energy(
None, hour, relative_power=self.ev_charge_hours[hour]
)
consumption += loaded_energy_ev
losses_wh_per_hour[hour_since_now] += verluste_eauto
if ev and ev_charge_hours[hour] > 0:
loaded_energy_ev, verluste_eauto = ev.charge_energy(
None, hour, relative_power=ev_charge_hours[hour]
)
consumption += loaded_energy_ev
losses_wh_per_hour[hour_idx] += verluste_eauto

# Process inverter logic
energy_feedin_grid_actual, energy_consumption_grid_actual, losses, eigenverbrauch = (
0.0,
0.0,
0.0,
0.0,
energy_feedin_grid_actual = energy_consumption_grid_actual = losses = eigenverbrauch = (
0.0
)
if self.battery:
self.battery.set_charge_allowed_for_hour(self.dc_charge_hours[hour], hour)
if self.inverter:
energy_produced = self.pv_prediction_wh[hour]

hour_ac_charge = ac_charge_hours[hour]
hour_dc_charge = dc_charge_hours[hour]
hourly_electricity_price = elect_price_hourly[hour]
hourly_energy_revenue = elect_revenue_per_hour_arr[hour]

battery.set_charge_allowed_for_hour(hour_dc_charge, hour)

if inverter:
energy_produced = pv_prediction_wh[hour]
(
energy_feedin_grid_actual,
energy_consumption_grid_actual,
losses,
eigenverbrauch,
) = self.inverter.process_energy(energy_produced, consumption, hour)
) = inverter.process_energy(energy_produced, consumption, hour)

# AC PV Battery Charge
if self.battery and self.ac_charge_hours[hour] > 0.0:
self.battery.set_charge_allowed_for_hour(1, hour)
battery_charged_energy_actual, battery_losses_actual = self.battery.charge_energy(
None, hour, relative_power=self.ac_charge_hours[hour]
if hour_ac_charge > 0.0:
battery.set_charge_allowed_for_hour(1, hour)
battery_charged_energy_actual, battery_losses_actual = battery.charge_energy(
None, hour, relative_power=hour_ac_charge
)
# print(hour, " ", battery_charged_energy_actual, " ",self.ac_charge_hours[hour]," ",self.battery.current_soc_percentage())
consumption += battery_charged_energy_actual
consumption += battery_losses_actual
energy_consumption_grid_actual += battery_charged_energy_actual
energy_consumption_grid_actual += battery_losses_actual
losses_wh_per_hour[hour_since_now] += battery_losses_actual

feedin_energy_per_hour[hour_since_now] = energy_feedin_grid_actual
consumption_energy_per_hour[hour_since_now] = energy_consumption_grid_actual
losses_wh_per_hour[hour_since_now] += losses
loads_energy_per_hour[hour_since_now] = consumption
electricity_price_per_hour[hour_since_now] = self.elect_price_hourly[hour]

total_battery_energy = battery_charged_energy_actual + battery_losses_actual
consumption += total_battery_energy
energy_consumption_grid_actual += total_battery_energy
losses_wh_per_hour[hour_idx] += battery_losses_actual

# Update hourly arrays
feedin_energy_per_hour[hour_idx] = energy_feedin_grid_actual
consumption_energy_per_hour[hour_idx] = energy_consumption_grid_actual
losses_wh_per_hour[hour_idx] += losses
loads_energy_per_hour[hour_idx] = consumption
electricity_price_per_hour[hour_idx] = hourly_electricity_price

# Financial calculations
costs_per_hour[hour_since_now] = (
energy_consumption_grid_actual * self.elect_price_hourly[hour]
)
revenue_per_hour[hour_since_now] = (
energy_feedin_grid_actual * self.elect_revenue_per_hour_arr[hour]
)
costs_per_hour[hour_idx] = energy_consumption_grid_actual * hourly_electricity_price
revenue_per_hour[hour_idx] = energy_feedin_grid_actual * hourly_energy_revenue

# Total cost and return
gesamtkosten_euro = np.nansum(costs_per_hour) - np.nansum(revenue_per_hour)
total_cost = np.nansum(costs_per_hour)
total_losses = np.nansum(losses_wh_per_hour)
total_revenue = np.nansum(revenue_per_hour)

# Prepare output dictionary
out: Dict[str, Union[np.ndarray, float]] = {
return {
"Last_Wh_pro_Stunde": loads_energy_per_hour,
"Netzeinspeisung_Wh_pro_Stunde": feedin_energy_per_hour,
"Netzbezug_Wh_pro_Stunde": consumption_energy_per_hour,
"Kosten_Euro_pro_Stunde": costs_per_hour,
"akku_soc_pro_stunde": soc_per_hour,
"Einnahmen_Euro_pro_Stunde": revenue_per_hour,
"Gesamtbilanz_Euro": gesamtkosten_euro,
"Gesamtbilanz_Euro": total_cost - total_revenue,
"EAuto_SoC_pro_Stunde": soc_ev_per_hour,
"Gesamteinnahmen_Euro": np.nansum(revenue_per_hour),
"Gesamtkosten_Euro": np.nansum(costs_per_hour),
"Gesamteinnahmen_Euro": total_revenue,
"Gesamtkosten_Euro": total_cost,
"Verluste_Pro_Stunde": losses_wh_per_hour,
"Gesamt_Verluste": np.nansum(losses_wh_per_hour),
"Gesamt_Verluste": total_losses,
"Home_appliance_wh_per_hour": home_appliance_wh_per_hour,
"Electricity_price": electricity_price_per_hour,
}

return out


# Initialize the Energy Management System, it is a singleton.
ems = EnergieManagementSystem()
Expand Down
2 changes: 1 addition & 1 deletion src/akkudoktoreos/utils/visualize.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ def add_chart_to_group(self, chart_func: Callable[[], None], title: str | None)
"""Add a chart function to the current group and save it as a PNG and SVG."""
self.current_group.append(chart_func)
if self.create_img and title:
server_output_dir = self.config.data_cache_path
server_output_dir = self.config.general.data_cache_path
server_output_dir.mkdir(parents=True, exist_ok=True)
fig, ax = plt.subplots()
chart_func()
Expand Down
Loading

0 comments on commit 480adf8

Please sign in to comment.