-
-
Notifications
You must be signed in to change notification settings - Fork 429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Standardising Benchmarks, with support for nanobench as an option for its backend #6448
Conversation
Can one of the admins verify this patch? |
Coverage summary from CodacySee diff coverage on Codacy
Coverage variation details
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: Diff coverage details
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: See your quality gate settings Change summary preferencesCodacy will stop sending the deprecated coverage status from June 5th, 2024. Learn more |
Congratulations for getting nanobench into the build system! |
Thanks for the response, but I am unable to see your suggestions in the PR. Could you please check once from your side? |
retest |
@vrnimje Thanks for making the changes for this to work. Unfortunately, our Jenkins CI is not working right now due to some issues on our side. I'll trigger the jobs once we've resolved the issue. Thanks for your patience. |
Ok, thanks for letting me know |
Coverage summary from CodacySee diff coverage on Codacy
Coverage variation details
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: Diff coverage details
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: See your quality gate settings Change summary preferences🚀 Don’t miss a bit, follow what’s new on Codacy. Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more |
retest lsu |
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
…x perf test Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
…minor changes to plotting script * Further simplified the default template for the benchmark Signed-off-by: Vedant <[email protected]>
…accordingly Signed-off-by: Vedant <[email protected]>
…orrect option Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
…, and made minor changes to perftests ci Signed-off-by: Vedant <[email protected]>
Signed-off-by: Vedant <[email protected]>
Performance test reportHPX PerformanceComparison
Info
Comparison
Info
Comparison
Info
Explanation of Symbols
|
tools/perftests_plot.py
Outdated
import sys | ||
import subprocess | ||
import json | ||
import matplotlib.pyplot as plt | ||
|
||
if len(sys.argv) != 2: | ||
if len(sys.argv) == 1: | ||
print("No benchmark selected!") | ||
else: | ||
print("Too many arguments!") | ||
print("Usage: python perftests_plot.py [path_to_benchmark_binary]") | ||
else: | ||
test_name = sys.argv[1] | ||
|
||
contents = subprocess.run([test_name, "--detailed_bench"], capture_output=True) | ||
|
||
json_obj = json.loads(contents.stdout.decode('utf-8')) | ||
|
||
test_names = [] | ||
samples = [] | ||
|
||
for test in json_obj["outputs"]: | ||
test_names.append(test["name"] + ",\n" + test["executor"]) | ||
samples.append(test["series"]) | ||
|
||
fig = plt.figure(figsize=(20, 6)) | ||
ax = fig.add_subplot() | ||
bp = ax.boxplot(samples, showfliers=False) | ||
plt.setp(ax.set_xticklabels(test_names), fontsize=7) | ||
plt.show() | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file can be removed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed in c35e4e4
CMakeLists.txt
Outdated
hpx_option( | ||
HPX_WITH_NANOBENCH | ||
BOOL | ||
"Use Nanobench for performance tests. Nanobench will be fetched using FetchContent (defualt: OFF)" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in c35e4e4
// std::cout << "-----Average-(hpx::small_vector<" << typeid(T).name() << ", " | ||
// << N << ">)------ \n" | ||
// << std::left << "Average execution time : " << std::right | ||
// << std::setw(8) << time / 1e9 << "\n"; | ||
// } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can be removed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed in c35e4e4
Signed-off-by: Vedant <[email protected]>
Performance test reportHPX PerformanceComparison
Info
Comparison
Info
Comparison
Info
Explanation of Symbols
|
@hkaiser Could we merge this? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
Background context: #6357
Proposed Changes
perftests_report
, which also supports nanobench as the backend