You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Oct 5, 2023. It is now read-only.
Within Zeno, some metrics are dependent on distill functions. For instance, in this example the avg_bert_score metric is dependent on the bert_score distill function.
However, it is possible that through human error, one might add a metric to ZenoParameters but forget to add the corresponding distill function, leading to a hard-to-debug error.
As a suggestion, what about adding an optional argument to specify dependencies as follows:
The key indicates the destination column of the distill function, and the value specifies the function itself. Then, the metric decorator could add an automatic check that the dependencies have been calculated before-hand and throw a comprehensible error if not.
The text was updated successfully, but these errors were encountered:
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Within Zeno, some metrics are dependent on distill functions. For instance, in this example the avg_bert_score metric is dependent on the bert_score distill function.
However, it is possible that through human error, one might add a metric to
ZenoParameters
but forget to add the corresponding distill function, leading to a hard-to-debug error.As a suggestion, what about adding an optional argument to specify dependencies as follows:
The key indicates the destination column of the distill function, and the value specifies the function itself. Then, the metric decorator could add an automatic check that the dependencies have been calculated before-hand and throw a comprehensible error if not.
The text was updated successfully, but these errors were encountered: