Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix calculation of attributes in statistics #128475

Merged
merged 9 commits into from
Oct 28, 2024

Conversation

gjohansson-ST
Copy link
Member

@gjohansson-ST gjohansson-ST commented Oct 15, 2024

Proposed change

Fix attributes for statistics
Besides calculating on each state change it also calculates on loading from recorder.
Checks that uom, device class and state class works together.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.

To help with the load of incoming pull requests:

@home-assistant
Copy link

Hey there @ThomDietrich, mind taking a look at this pull request as it has been labeled with an integration (statistics) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of statistics can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign statistics Removes the current integration label and assignees on the pull request, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

@gjohansson-ST gjohansson-ST marked this pull request as draft October 15, 2024 19:02
@gjohansson-ST gjohansson-ST marked this pull request as ready for review October 16, 2024 21:27
@frenck frenck added this to the 2024.11.0b0 milestone Oct 18, 2024
@frenck frenck added the smash Indicator this PR is close to finish for merging or closing label Oct 18, 2024
@ThomDietrich
Copy link
Contributor

Hey @gjohansson-ST just a few comments from my side. Please do not see them as a general pushback as there are a few good ideas in here! Thanks for you contribution :)

@ThomDietrich
Copy link
Contributor

Hey @gjohansson-ST, besides those random code changes, I'm having a hard time to understand what the improvement is.

Fix attributes for statistics

What did you fix?

Besides calculating on each state change it also calculates on loading from recorder.

Maybe not necessary but also didn't hurt. Great if you were able to boost performance.

Checks that uom, device class and state class works together.

What does that mean?

@gjohansson-ST
Copy link
Member Author

Hey @gjohansson-ST, besides those random code changes, I'm having a hard time to understand what the improvement is.

So the short story is that users had problems and issues were raised for helpers which had flopping state attributes.
As example if the sensor is loaded without those attributes (loaded from recorder) but then the state attributes are loaded later it would raise an issue for them. Also needs to check that UoM matches what the device class accept, we only need to act on proper numeric state changes etc. etc.

In the end we should have a consistent behavior between the helpers on how this works so the experience for the user is the same regardless if they use statistics, group, min_max etc. therefore I also rewrite some parts to align it between those helpers.

@emontnemery
Copy link
Contributor

@gjohansson-ST the PR has conflicts

@gjohansson-ST gjohansson-ST force-pushed the statistics-uom-load-from-db branch from 3e0ec28 to 087e8e3 Compare October 26, 2024 13:47
Copy link
Contributor

@ThomDietrich ThomDietrich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey! Good addition overall and I like the extensive test cases.
My question would be whether any of those test case covered issues should additionally raise a warning, e.g. in logs or as a repairs entry. No need to cover this in the PR here

homeassistant/components/statistics/sensor.py Outdated Show resolved Hide resolved
homeassistant/components/statistics/sensor.py Outdated Show resolved Hide resolved
if (
device_class in DEVICE_CLASS_UNITS
and unit not in DEVICE_CLASS_UNITS[device_class]
):
return None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you have diversified the cases returning None, did you consider that any of those cases should instead create a warning? I myself are not sure...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe. I'm a bit hesitant given the UoM is calculated so it might bring warnings/issues which is confusing and don't make sense to the user.
Let's consider in a follow-up in that case

homeassistant/components/statistics/sensor.py Show resolved Hide resolved
homeassistant/components/statistics/sensor.py Outdated Show resolved Hide resolved

def _derive_unit_of_measurement(self, new_state: State) -> str | None:
def _calculate_state_attributes(self, new_state: State) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In light of this new function I suggest to rename the existing function _update_attributes(self). Those names clash currently. How about something like _update_additional_attributes(self) or _calculate_custom_attributes(self)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should update both _update_attributes and the attributes dict to make it clear it's about extra state attributes.
Maybe that can happen in a separate PR, to not pollute this PR?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, let's "clean" this in a follow-up PR.
I didn't even consider self.attributes nor the _update_attributes method

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like an important change for clarity, but good on the extra PR

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@emontnemery emontnemery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion for updating docstrings

homeassistant/components/statistics/sensor.py Outdated Show resolved Hide resolved
homeassistant/components/statistics/sensor.py Outdated Show resolved Hide resolved
homeassistant/components/statistics/sensor.py Outdated Show resolved Hide resolved
@gjohansson-ST gjohansson-ST force-pushed the statistics-uom-load-from-db branch from 087e8e3 to 2d3bad0 Compare October 28, 2024 14:36
Copy link
Contributor

@emontnemery emontnemery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks @gjohansson-ST 👍

I think the remaining open comments can be addressed in follow-up PRs, @ThomDietrich ?

Copy link
Contributor

@ThomDietrich ThomDietrich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes all good and settled from my side.
Thanks for your work and for continuing the statistics component! 💚

@gjohansson-ST gjohansson-ST merged commit 7d699c6 into dev Oct 28, 2024
32 checks passed
@gjohansson-ST gjohansson-ST deleted the statistics-uom-load-from-db branch October 28, 2024 18:45
@github-actions github-actions bot locked and limited conversation to collaborators Oct 29, 2024
@frenck frenck removed this from the 2024.11.0b0 milestone Nov 5, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants