Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmarks: add metrics benchmark #825

Merged
merged 1 commit into from
Oct 30, 2024

Conversation

iRevive
Copy link
Contributor

@iRevive iRevive commented Oct 29, 2024

Yet another part of the #786.

Baseline

Measurment: average time, lower is better.

SDK is roughly 15% to 30% slower than the OtelJava.

Benchmark                      (backend)       (operation)     (variation)  Mode  Cnt      Score      Error   Units

eightThreadsCommonLabelSet      oteljava      long_counter            noop  avgt   10  28638.180 ±  3617.497   ns/op
eightThreadsCommonLabelSet      oteljava      long_counter  sdk_cumulative  avgt   10  27742.687 ±  1200.754   ns/op
eightThreadsCommonLabelSet      oteljava      long_counter       sdk_delta  avgt   10  28586.511 ±  2407.448   ns/op
eightThreadsCommonLabelSet           sdk      long_counter            noop  avgt   10  26353.959 ±   771.915   ns/op
eightThreadsCommonLabelSet           sdk      long_counter  sdk_cumulative  avgt   10  35163.158 ±   375.981   ns/op
eightThreadsCommonLabelSet           sdk      long_counter       sdk_delta  avgt   10  34662.634 ±   360.655   ns/op

eightThreadsSeparateLabelSets   oteljava      long_counter            noop  avgt   10  26951.976 ±  2104.885   ns/op
eightThreadsSeparateLabelSets   oteljava      long_counter  sdk_cumulative  avgt   10  28151.917 ±  3897.626   ns/op
eightThreadsSeparateLabelSets   oteljava      long_counter       sdk_delta  avgt   10  26909.918 ±   571.061   ns/op
eightThreadsSeparateLabelSets        sdk      long_counter            noop  avgt   10  27774.570 ±  2493.445   ns/op
eightThreadsSeparateLabelSets        sdk      long_counter  sdk_cumulative  avgt   10  35102.793 ±   784.706   ns/op
eightThreadsSeparateLabelSets        sdk      long_counter       sdk_delta  avgt   10  34650.908 ±   320.706   ns/op

oneThread                       oteljava      long_counter            noop  avgt   10   4948.533 ±    81.298   ns/op
oneThread                       oteljava      long_counter  sdk_cumulative  avgt   10   5326.867 ±   277.532   ns/op
oneThread                       oteljava      long_counter       sdk_delta  avgt   10   5255.243 ±   122.588   ns/op
oneThread                            sdk      long_counter            noop  avgt   10   5120.052 ±   223.250   ns/op
oneThread                            sdk      long_counter  sdk_cumulative  avgt   10   6962.334 ±   266.006   ns/op
oneThread                            sdk      long_counter       sdk_delta  avgt   10   7658.263 ±   969.198   ns/op

recordToMultipleAttributes      oteljava      long_counter            noop  avgt   10   5469.453 ±   436.134   ns/op
recordToMultipleAttributes      oteljava      long_counter  sdk_cumulative  avgt   10   7049.623 ±   897.224   ns/op
recordToMultipleAttributes      oteljava      long_counter       sdk_delta  avgt   10   8618.673 ±  4704.295   ns/op
recordToMultipleAttributes           sdk      long_counter            noop  avgt   10   5533.041 ±   118.798   ns/op
recordToMultipleAttributes           sdk      long_counter  sdk_cumulative  avgt   10  11766.431 ±   226.243   ns/op
recordToMultipleAttributes           sdk      long_counter       sdk_delta  avgt   10  11744.749 ±  1033.673   ns/op

This PR

SDK is only 1-10% slower (depending on the number of attributes and concurrency levels).

Benchmark                      (backend)       (operation)     (variation)  Mode  Cnt      Score      Error   Units

eightThreadsCommonLabelSet      oteljava      long_counter            noop  avgt   10  27182.080 ± 2064.094   ns/op
eightThreadsCommonLabelSet      oteljava      long_counter  sdk_cumulative  avgt   10  27002.882 ±  624.538   ns/op
eightThreadsCommonLabelSet      oteljava      long_counter       sdk_delta  avgt   10  26991.866 ±  551.068   ns/op
eightThreadsCommonLabelSet           sdk      long_counter            noop  avgt   10  26568.445 ±  559.356   ns/op
eightThreadsCommonLabelSet           sdk      long_counter  sdk_cumulative  avgt   10  26970.777 ±  534.541   ns/op
eightThreadsCommonLabelSet           sdk      long_counter       sdk_delta  avgt   10  28479.440 ± 3924.062   ns/op

eightThreadsSeparateLabelSets   oteljava      long_counter            noop  avgt   10  31506.592 ± 6814.522   ns/op
eightThreadsSeparateLabelSets   oteljava      long_counter  sdk_cumulative  avgt   10  28212.355 ± 2481.786   ns/op
eightThreadsSeparateLabelSets   oteljava      long_counter       sdk_delta  avgt   10  27082.346 ±  544.882   ns/op
eightThreadsSeparateLabelSets        sdk      long_counter            noop  avgt   10  26845.652 ±  646.818   ns/op
eightThreadsSeparateLabelSets        sdk      long_counter  sdk_cumulative  avgt   10  27930.532 ± 2891.188   ns/op
eightThreadsSeparateLabelSets        sdk      long_counter       sdk_delta  avgt   10  27270.965 ±  606.613   ns/op

oneThread                       oteljava      long_counter            noop  avgt   10   4817.086 ±   47.913   ns/op
oneThread                       oteljava      long_counter  sdk_cumulative  avgt   10   5373.915 ±  165.448   ns/op
oneThread                       oteljava      long_counter       sdk_delta  avgt   10   5137.084 ±   50.088   ns/op
oneThread                            sdk      long_counter            noop  avgt   10   4912.818 ±  295.623   ns/op
oneThread                            sdk      long_counter  sdk_cumulative  avgt   10   5320.152 ±  261.549   ns/op
oneThread                            sdk      long_counter       sdk_delta  avgt   10   5770.930 ±  172.216   ns/op

recordToMultipleAttributes      oteljava      long_counter            noop  avgt   10   5160.638 ±  566.987   ns/op
recordToMultipleAttributes      oteljava      long_counter  sdk_cumulative  avgt   10   5922.355 ±  199.489   ns/op
recordToMultipleAttributes      oteljava      long_counter       sdk_delta  avgt   10   6391.997 ±  249.286   ns/op
recordToMultipleAttributes           sdk      long_counter            noop  avgt   10   5460.340 ±  246.778   ns/op
recordToMultipleAttributes           sdk      long_counter  sdk_cumulative  avgt   10   7095.079 ±  156.089   ns/op
recordToMultipleAttributes           sdk      long_counter       sdk_delta  avgt   10   7142.590 ±  174.927   ns/op

@iRevive iRevive added module:sdk Features and improvements to the sdk module performance Performance improvements labels Oct 29, 2024
@iRevive iRevive merged commit bb56bff into typelevel:main Oct 30, 2024
10 checks passed
@iRevive iRevive deleted the benchmarks/metrics branch October 30, 2024 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module:sdk Features and improvements to the sdk module performance Performance improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant