Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(validator): support validation config to map actual and predicted #1773

Merged
merged 3 commits into from
Sep 10, 2024

Conversation

sthaha
Copy link
Collaborator

@sthaha sthaha commented Sep 9, 2024

This PR introduces the following changes to validator

  • Use predicted instead of expected terminology to align with MAPE and MSE teminology
  • Add ability map actual and predicted in validations. This allwos for
config:
  mapping:
     actual: metal
     predicted: vm

validations:
  - metal: <metal-query>
    vm: <vm-query>
  • Generated reports and artifacts now make use of these label
  • Add support for Units
image

@sthaha sthaha requested a review from KaiyiLiu1234 September 9, 2024 23:59
Copy link
Contributor

github-actions bot commented Sep 9, 2024

🤖 SeineSailor

Here is a concise summary of the pull request changes:

Summary: This pull request refactors the validator module, introducing changes to the ValidationResult class and related functions. Key modifications include:

  • Renaming expected to predicted to align with MAPE and MSE terminology
  • Adding new attributes for actual_label, predicted_label, and units to support mapping actual and predicted values in validations
  • Updating test functions for MSE to use predicted instead of expected
  • Increasing the maximum series length for MSE tests to 1000

Impact: These changes affect the internal structure and behavior of the validator module, but do not impact the external interface or functionality, as long as users update their code to use the new attribute names.

Observations:

  • The changes seem to be focused on improving the internal implementation of the validator module, which is a good practice for maintainability and scalability.
  • The updated terminology and added features will likely improve the accuracy and usefulness of validation results.
  • It would be beneficial to include documentation updates to reflect the changes, ensuring that users are aware of the new attribute names and their usage.

Overall, this pull request appears to be a well-structured and thoughtful refactoring effort, with a focus on improving the internal implementation and functionality of the validator module.

This commit
* renames `expected` as `predicted` to use consistent naming every
  where (including MSE and MAPE)

* adds support for defining optional config.mapping for
mapping actual and predicted labels.

* Adds support of `units` to be added to validations

This allows validations to be written as follows

```
config
  mapping:
    actual: metal
    predicted: vm

validations:
  - metal: <metal-query>
    vm: <vm-query>
    units: Watts

```

Signed-off-by: Sunil Thaha <[email protected]>
@sthaha sthaha force-pushed the feat-validator-mappings branch from 36c1994 to de99d53 Compare September 10, 2024 00:10
Copy link
Collaborator

@KaiyiLiu1234 KaiyiLiu1234 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@sthaha sthaha merged commit ac5ee8b into sustainable-computing-io:main Sep 10, 2024
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants