add label to metrics #34
Conversation
rekcurd/utils/__init__.py
Outdated
| if num is None: | ||
| self.num = 0 | ||
| self.accuracy = 0.0 | ||
| self.precision = [0.0] | ||
| self.recall = [0.0] | ||
| self.fvalue = [0.0] | ||
| self.option = {} | ||
| self.label = [0.0] |
There was a problem hiding this comment.
Is it OK to set list[float] here?
There was a problem hiding this comment.
Yes.PredictLabel accepts List[float] and IO (grpc message)'s tensor's val is repeated float .
There was a problem hiding this comment.
I mean that the return type of label depends on the user's predictor as I wrote on rekcurd_worker_servicer.py.
There was a problem hiding this comment.
@keigohtr
I removed default None value from EvaluateResult!
Codecov Report
@@ Coverage Diff @@
## master #34 +/- ##
==========================================
+ Coverage 81.17% 81.92% +0.74%
==========================================
Files 14 14
Lines 712 708 -4
==========================================
+ Hits 578 580 +2
+ Misses 134 128 -6
Continue to review full report at Codecov.
|
| self.option = option | ||
| def __init__(self, num: int, accuracy: float, precision: List[float], | ||
| recall: List[float], fvalue: List[float], label: List[PredictLabel], | ||
| option: Dict[str, float] = {}): |
There was a problem hiding this comment.
BTW, I think option type is just dict not Dict[str, float]...?
There was a problem hiding this comment.
It's OK at the moment.
Could you make an issue for this?
Additional metrics must be a general field for ML evaluation. Current spec specifies our use-case, so we need to fix it.
What is this PR for?
added label to
EvaluationMetricsto identify which precision, recall and fvalue correspond to which labels.This PR includes
EvaluateResultclassrekcurd_pb2.EvaluationMetricsWhat type of PR is it?
Feature
What is the issue?
N/A
How should this be tested?
python -m unittest test.test_dashboard_servicer