Implements feature based perf instrumentation comparision #2642

Sign in to view logs

Triggered via pull request July 9, 2023 11:45

vulder

synchronize #806

f-StaticAnalysisMotivatedSynthBenchmarksImpl

Status Success

Total duration 24s

Artifacts –

This run and associated checks have been archived and are scheduled for deletion. Learn more about checks retention

pre-commit.yml

on: pull_request

Annotations

21 errors and 24 warnings

[mypy] varats-core/varats/report/tef_report.py#L172: varats-core/varats/report/tef_report.py#L172

error: Function is missing a return type annotation [no-untyped-def]

[mypy] varats-core/varats/report/tef_report.py#L172: varats-core/varats/report/tef_report.py#L172

note: Use "-> None" if function does not return a value

[mypy] varats/varats/experiments/vara/feature_perf_precision.py#L93: varats/varats/experiments/vara/feature_perf_precision.py#L93

error: Class cannot subclass "ProjectStep" (has type "Any") [misc]

[mypy] varats/varats/experiments/vara/feature_perf_precision.py#L113: varats/varats/experiments/vara/feature_perf_precision.py#L113

error: Class cannot subclass "ProjectStep" (has type "Any") [misc]

[mypy] varats/varats/experiments/vara/feature_perf_precision.py#L150: varats/varats/experiments/vara/feature_perf_precision.py#L150

error: Function is missing a type annotation for one or more arguments [no-untyped-def]

[mypy] varats/varats/experiments/vara/feature_perf_precision.py#L226: varats/varats/experiments/vara/feature_perf_precision.py#L226

error: Non-overlapping identity check (left operand type: "str", right operand type: "Literal[ProjectDomains.TEST]") [comparison-overlap]

[mypy] varats/varats/experiments/vara/feature_perf_precision.py#L293: varats/varats/experiments/vara/feature_perf_precision.py#L293

error: Function is missing a type annotation for one or more arguments [no-untyped-def]

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L124 <301>: varats/varats/experiments/vara/feature_perf_precision.py#L124

Line too long (104/80) (line-too-long)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L270 <301>: varats/varats/experiments/vara/feature_perf_precision.py#L270

Line too long (81/80) (line-too-long)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L403 <301>: varats/varats/experiments/vara/feature_perf_precision.py#L403

Line too long (81/80) (line-too-long)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L49 <115>: varats/varats/experiments/vara/feature_perf_precision.py#L49

Missing class docstring (missing-class-docstring)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L77 <115>: varats/varats/experiments/vara/feature_perf_precision.py#L77

Missing class docstring (missing-class-docstring)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L85 <115>: varats/varats/experiments/vara/feature_perf_precision.py#L85

Missing class docstring (missing-class-docstring)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L93 <115>: varats/varats/experiments/vara/feature_perf_precision.py#L93

Missing class docstring (missing-class-docstring)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L113 <115>: varats/varats/experiments/vara/feature_perf_precision.py#L113

Missing class docstring (missing-class-docstring)

[pylint] varats/varats/tables/feature_perf_precision.py#L65 <325>: varats/varats/tables/feature_perf_precision.py#L65

Unnecessary parens after 'if' keyword (superfluous-parens)

[pylint] varats/varats/tables/feature_perf_precision.py#L67 <325>: varats/varats/tables/feature_perf_precision.py#L67

Unnecessary parens after 'elif' keyword (superfluous-parens)

[pylint] varats/varats/tables/feature_perf_precision.py#L434 <301>: varats/varats/tables/feature_perf_precision.py#L434

Line too long (155/80) (line-too-long)

[pylint] varats/varats/tables/feature_perf_precision.py#L112 <103>: varats/varats/tables/feature_perf_precision.py#L112

Variable name "gt" doesn't conform to snake_case naming style (invalid-name)

[pylint] varats/varats/tables/feature_perf_precision.py#L215 <116>: varats/varats/tables/feature_perf_precision.py#L215

Missing function or method docstring (missing-function-docstring)

[pylint] varats/varats/tables/feature_perf_precision.py#L344 <914>: varats/varats/tables/feature_perf_precision.py#L344

Too many local variables (24/20) (too-many-locals)

[pylint] varats/varats/projects/perf_tests/feature_perf_cs_collection.py#L202 <511>: varats/varats/projects/perf_tests/feature_perf_cs_collection.py#L202

TODO: fix with commit after merge (fixme)

[pylint] varats/varats/projects/perf_tests/feature_perf_cs_collection.py#L264 <511>: varats/varats/projects/perf_tests/feature_perf_cs_collection.py#L264

TODO: fix with commit after merge (fixme)

[pylint] varats/varats/projects/perf_tests/feature_perf_cs_collection.py#L329 <511>: varats/varats/projects/perf_tests/feature_perf_cs_collection.py#L329

TODO: fix with commit after merge (fixme)

[pylint] varats/varats/projects/perf_tests/feature_perf_cs_collection.py#L392 <511>: varats/varats/projects/perf_tests/feature_perf_cs_collection.py#L392

TODO: fix with commit after merge (fixme)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L58 <511>: varats/varats/experiments/vara/feature_perf_precision.py#L58

TODO: clean up (fixme)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L12 <611>: varats/varats/experiments/vara/feature_perf_precision.py#L12

Unused Step imported from benchbuild.utils.actions (unused-import)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L12 <611>: varats/varats/experiments/vara/feature_perf_precision.py#L12

Unused Compile imported from benchbuild.utils.actions (unused-import)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L12 <611>: varats/varats/experiments/vara/feature_perf_precision.py#L12

Unused Clean imported from benchbuild.utils.actions (unused-import)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L22 <611>: varats/varats/experiments/vara/feature_perf_precision.py#L22

Unused ExperimentHandle imported from varats.experiment.experiment_util (unused-import)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L22 <611>: varats/varats/experiments/vara/feature_perf_precision.py#L22

Unused VersionExperiment imported from varats.experiment.experiment_util (unused-import)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L34 <611>: varats/varats/experiments/vara/feature_perf_precision.py#L34

Unused RunVaRATracedWorkloads imported from varats.experiments.vara.feature_experiment (unused-import)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L34 <611>: varats/varats/experiments/vara/feature_perf_precision.py#L34

Unused RunVaRATracedXRayWorkloads imported from varats.experiments.vara.feature_experiment (unused-import)

[pylint] varats/varats/experiments/vara/feature_perf_precision.py#L45 <611>: varats/varats/experiments/vara/feature_perf_precision.py#L45

Unused TEFReport imported from varats.report.tef_report (unused-import)

[pylint] varats/varats/tables/feature_perf_precision.py#L140 <511>: varats/varats/tables/feature_perf_precision.py#L140

TODO: double check ttest handling (fixme)

[pylint] varats/varats/tables/feature_perf_precision.py#L296 <511>: varats/varats/tables/feature_perf_precision.py#L296

TODO: check, maybe we need a "very small value cut off" (fixme)

[pylint] varats/varats/tables/feature_perf_precision.py#L304 <511>: varats/varats/tables/feature_perf_precision.py#L304

TODO: how to handle this? (fixme)

[pylint] varats/varats/tables/feature_perf_precision.py#L371 <511>: varats/varats/tables/feature_perf_precision.py#L371

TODO: multiple patch cycles (fixme)

[pylint] varats/varats/tables/feature_perf_precision.py#L3 <611>: varats/varats/tables/feature_perf_precision.py#L3

Unused import shutil (unused-import)

[pylint] varats/varats/tables/feature_perf_precision.py#L4 <611>: varats/varats/tables/feature_perf_precision.py#L4

Unused import tempfile (unused-import)

[pylint] varats/varats/tables/feature_perf_precision.py#L7 <611>: varats/varats/tables/feature_perf_precision.py#L7

Unused Path imported from pathlib (unused-import)

[pylint] varats/varats/tables/feature_perf_precision.py#L11 <611>: varats/varats/tables/feature_perf_precision.py#L11

Unused NoEscape imported from pylatex (unused-import)

[pylint] varats/varats/tables/feature_perf_precision.py#L16 <611>: varats/varats/tables/feature_perf_precision.py#L16

Unused FeaturePerfRunner imported from varats.experiments.vara.feature_perf_runner (unused-import)

[pylint] varats/varats/tables/feature_perf_precision.py#L17 <611>: varats/varats/tables/feature_perf_precision.py#L17

Unused load_tef_report imported from varats.jupyterhelper.file (unused-import)

[pylint] varats/varats/tables/feature_perf_precision.py#L21 <611>: varats/varats/tables/feature_perf_precision.py#L21

Unused TimeReportAggregate imported from varats.report.gnu_time_report (unused-import)