-
Notifications
You must be signed in to change notification settings - Fork 40
/
benchmark.txt
294 lines (282 loc) · 21.1 KB
/
benchmark.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------< dev.openfeature:sdk >-------------------------
[INFO] Building OpenFeature Java SDK 1.12.1
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] Parameter 'encoding' is unknown for plugin 'maven-checkstyle-plugin:3.5.0:check (validate)'
[WARNING] Parameter 'encoding' is unknown for plugin 'maven-checkstyle-plugin:3.5.0:check (validate)'
[WARNING] Parameter 'encoding' is unknown for plugin 'maven-checkstyle-plugin:3.5.0:check (validate)'
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ sdk ---
[INFO] Deleting /home/todd/git/java-sdk/target
[INFO]
[INFO] --- checkstyle:3.5.0:check (validate) @ sdk ---
[INFO] Starting audit...
Audit done.
[INFO] You have 0 Checkstyle violations.
[INFO]
[INFO] --- jacoco:0.8.12:prepare-agent (prepare-agent) @ sdk ---
[INFO] surefireArgLine set to -javaagent:/home/todd/.m2/repository/org/jacoco/org.jacoco.agent/0.8.12/org.jacoco.agent-0.8.12-runtime.jar=destfile=/home/todd/git/java-sdk/target/coverage-reports/jacoco-ut.exec
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ sdk ---
[INFO] skip non existing resourceDirectory /home/todd/git/java-sdk/src/main/resources
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ sdk ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 65 source files with javac [debug target 1.8] to target/classes
[WARNING] bootstrap class path not set in conjunction with -source 8
[WARNING] source value 8 is obsolete and will be removed in a future release
[WARNING] target value 8 is obsolete and will be removed in a future release
[WARNING] To suppress warnings about obsolete options, use -Xlint:-options.
[INFO] Annotation processing is enabled because one or more processors were found
on the class path. A future release of javac may disable annotation processing
unless at least one processor is specified by name (-processor), or a search
path is specified (--processor-path, --processor-module-path), or annotation
processing is enabled explicitly (-proc:only, -proc:full).
Use -Xlint:-options to suppress this message.
Use -proc:none to disable annotation processing.
[WARNING] /home/todd/git/java-sdk/src/main/java/dev/openfeature/sdk/MutableStructure.java:[19,1] Generating equals/hashCode implementation but without a call to superclass, even though this class does not extend java.lang.Object. If this is intentional, add '@EqualsAndHashCode(callSuper=false)' to your type.
[WARNING] /home/todd/git/java-sdk/src/main/java/dev/openfeature/sdk/ImmutableStructure.java:[22,1] Generating equals/hashCode implementation but without a call to superclass, even though this class does not extend java.lang.Object. If this is intentional, add '@EqualsAndHashCode(callSuper=false)' to your type.
[WARNING] /home/todd/git/java-sdk/src/main/java/dev/openfeature/sdk/EventDetails.java:[9,1] Generating equals/hashCode implementation but without a call to superclass, even though this class does not extend java.lang.Object. If this is intentional, add '@EqualsAndHashCode(callSuper=false)' to your type.
[WARNING] /home/todd/git/java-sdk/src/main/java/dev/openfeature/sdk/Value.java:[27,26] finalize() in java.lang.Object has been deprecated and marked for removal
[INFO] /home/todd/git/java-sdk/src/main/java/dev/openfeature/sdk/NoOpProvider.java: Some input files use or override a deprecated API.
[INFO] /home/todd/git/java-sdk/src/main/java/dev/openfeature/sdk/NoOpProvider.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/todd/git/java-sdk/src/main/java/dev/openfeature/sdk/Value.java: Some input files use unchecked or unsafe operations.
[INFO] /home/todd/git/java-sdk/src/main/java/dev/openfeature/sdk/Value.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- checkstyle:3.5.0:check (validate) @ sdk ---
[INFO] Starting audit...
Audit done.
[INFO] You have 0 Checkstyle violations.
[INFO]
[INFO] --- jacoco:0.8.12:prepare-agent (prepare-agent) @ sdk ---
[INFO] surefireArgLine set to -javaagent:/home/todd/.m2/repository/org/jacoco/org.jacoco.agent/0.8.12/org.jacoco.agent-0.8.12-runtime.jar=destfile=/home/todd/git/java-sdk/target/coverage-reports/jacoco-ut.exec
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ sdk ---
[INFO] skip non existing resourceDirectory /home/todd/git/java-sdk/src/main/resources
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ sdk ---
[INFO] Nothing to compile - all classes are up to date.
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ sdk ---
[INFO] Copying 2 resources from src/test/resources to target/test-classes
[INFO]
[INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ sdk ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 52 source files with javac [debug target 1.8] to target/test-classes
[WARNING] bootstrap class path not set in conjunction with -source 8
[WARNING] source value 8 is obsolete and will be removed in a future release
[WARNING] target value 8 is obsolete and will be removed in a future release
[WARNING] To suppress warnings about obsolete options, use -Xlint:-options.
[INFO] Annotation processing is enabled because one or more processors were found
on the class path. A future release of javac may disable annotation processing
unless at least one processor is specified by name (-processor), or a search
path is specified (--processor-path, --processor-module-path), or annotation
processing is enabled explicitly (-proc:only, -proc:full).
Use -Xlint:-options to suppress this message.
Use -proc:none to disable annotation processing.
[INFO] /home/todd/git/java-sdk/src/test/java/dev/openfeature/sdk/EventsTest.java: Some input files use or override a deprecated API.
[INFO] /home/todd/git/java-sdk/src/test/java/dev/openfeature/sdk/EventsTest.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/todd/git/java-sdk/src/test/java/dev/openfeature/sdk/HookSpecTest.java: Some input files use unchecked or unsafe operations.
[INFO] /home/todd/git/java-sdk/src/test/java/dev/openfeature/sdk/HookSpecTest.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] >>> jmh:0.2.2:benchmark (default-cli) > process-test-resources @ sdk >>>
[INFO]
[INFO] --- checkstyle:3.5.0:check (validate) @ sdk ---
[INFO] Starting audit...
Audit done.
[INFO] You have 0 Checkstyle violations.
[INFO]
[INFO] --- jacoco:0.8.12:prepare-agent (prepare-agent) @ sdk ---
[INFO] surefireArgLine set to -javaagent:/home/todd/.m2/repository/org/jacoco/org.jacoco.agent/0.8.12/org.jacoco.agent-0.8.12-runtime.jar=destfile=/home/todd/git/java-sdk/target/coverage-reports/jacoco-ut.exec
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ sdk ---
[INFO] skip non existing resourceDirectory /home/todd/git/java-sdk/src/main/resources
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ sdk ---
[INFO] Nothing to compile - all classes are up to date.
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ sdk ---
[INFO] Copying 2 resources from src/test/resources to target/test-classes
[INFO]
[INFO] <<< jmh:0.2.2:benchmark (default-cli) < process-test-resources @ sdk <<<
[INFO]
[INFO]
[INFO] --- jmh:0.2.2:benchmark (default-cli) @ sdk ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 52 source files to /home/todd/git/java-sdk/target/test-classes
[INFO] /home/todd/git/java-sdk/src/test/java/dev/openfeature/sdk/LockingTest.java: Some input files use or override a deprecated API.
[INFO] /home/todd/git/java-sdk/src/test/java/dev/openfeature/sdk/LockingTest.java: Recompile with -Xlint:deprecation for details.
[INFO] /home/todd/git/java-sdk/src/test/java/dev/openfeature/sdk/internal/TriConsumerTest.java: Some input files use unchecked or unsafe operations.
[INFO] /home/todd/git/java-sdk/src/test/java/dev/openfeature/sdk/internal/TriConsumerTest.java: Recompile with -Xlint:unchecked for details.
[INFO] Executing the JMH benchmarks
# JMH version: 1.37
# VM version: JDK 21.0.4, OpenJDK 64-Bit Server VM, 21.0.4+7
# VM invoker: /usr/lib/jvm/java-21-openjdk/bin/java
# VM options: -Xmx1024m -XX:+UnlockExperimentalVMOptions -XX:+UseEpsilonGC
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
# Warmup: <none>
# Measurement: 1 iterations, single-shot each
# Timeout: 10 min per iteration
# Threads: 1 thread
# Benchmark mode: Single shot invocation time
# Benchmark: dev.openfeature.sdk.benchmark.AllocationBenchmark.run
# Run progress: 0.00% complete, ETA 00:00:00
# Fork: 1 of 1
[0.001s][warning][gc,init] Consider setting -Xms equal to -Xmx to avoid resizing hiccups
[0.001s][warning][gc,init] Consider enabling -XX:+AlwaysPreTouch to avoid memory commit hiccups
Iteration 1: num #instances #bytes class name (module)
-------------------------------------------------------
1: 480234 23051232 java.util.HashMap ([email protected])
2: 150497 12050088 [Ljava.util.HashMap$Node; ([email protected])
3: 332017 10624544 java.util.HashMap$Node ([email protected])
4: 47815 9732480 [B ([email protected])
5: 305991 8105872 [Ljava.lang.Object; ([email protected])
6: 366682 5866912 java.util.Optional ([email protected])
7: 183332 5866624 java.util.HashMap$EntryIterator ([email protected])
8: 172970 5535040 java.util.Collections$UnmodifiableMap ([email protected])
9: 100000 4000000 dev.openfeature.sdk.HookContext
10: 100000 4000000 dev.openfeature.sdk.HookContext$HookContextBuilder
11: 230006 3680096 dev.openfeature.sdk.Value
12: 200062 3200992 java.util.HashMap$EntrySet ([email protected])
13: 132870 3188880 java.util.ArrayList ([email protected])
14: 192292 3076672 dev.openfeature.sdk.ImmutableStructure
15: 182292 2916672 dev.openfeature.sdk.ImmutableContext
16: 50000 2000000 dev.openfeature.sdk.FlagEvaluationDetails
17: 50000 2000000 dev.openfeature.sdk.ProviderEvaluation
18: 122968 1967488 java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet ([email protected])
19: 149 1884376 [Ljdk.internal.vm.FillerElement; ([email protected])
20: 56476 1807232 java.util.ArrayList$Itr ([email protected])
21: 37481 1799088 dev.openfeature.sdk.FlagEvaluationDetails$FlagEvaluationDetailsBuilder
22: 100001 1600016 dev.openfeature.sdk.NoOpProvider$$Lambda/0x000076e79c02fa78
23: 50000 1600000 [Ldev.openfeature.sdk.EvaluationContext;
24: 50000 1600000 [Ljava.util.List; ([email protected])
25: 100000 1600000 dev.openfeature.sdk.OpenFeatureClient$$Lambda/0x000076e79c082800
26: 36720 1468800 dev.openfeature.sdk.ProviderEvaluation$ProviderEvaluationBuilder
27: 87481 1399696 dev.openfeature.sdk.ImmutableMetadata
28: 50000 1200000 dev.openfeature.sdk.FlagEvaluationOptions
29: 74201 1187216 dev.openfeature.sdk.ImmutableMetadata$ImmutableMetadataBuilder
30: 73235 1171760 java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$UnmodifiableEntry ([email protected])
31: 45869 1100856 java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$1 ([email protected])
32: 43776 1050624 dev.openfeature.sdk.FlagEvaluationOptions$FlagEvaluationOptionsBuilder
33: 40016 960384 dev.openfeature.sdk.HookSupport$$Lambda/0x000076e79c081b78
34: 39967 959208 dev.openfeature.sdk.HookSupport$$Lambda/0x000076e79c081da8
35: 57783 924528 dev.openfeature.sdk.internal.AutoCloseableReentrantReadWriteLock$$Lambda/0x000076e79c02eae8
36: 4490 721440 [I ([email protected])
37: 26594 638256 java.lang.String ([email protected])
38: 1461 390008 [J ([email protected])
39: 2361 288784 java.lang.Class ([email protected])
40: 4632 259392 jdk.internal.org.objectweb.asm.SymbolTable$Entry ([email protected])
41: 10001 240024 java.lang.Double ([email protected])
42: 2502 180144 java.lang.reflect.Field ([email protected])
43: 6007 144168 java.lang.StringBuilder ([email protected])
44: 180 140968 [Ljdk.internal.org.objectweb.asm.SymbolTable$Entry; ([email protected])
45: 3827 122464 java.util.concurrent.ConcurrentHashMap$Node ([email protected])
46: 48 122168 [C ([email protected])
47: 1440 113512 [S ([email protected])
48: 1201 105688 java.lang.reflect.Method ([email protected])
49: 3031 79600 [Ljava.lang.Class; ([email protected])
50: 1351 75656 jdk.internal.org.objectweb.asm.Label ([email protected])
51: 1561 74928 java.lang.invoke.MemberName ([email protected])
52: 334 74816 jdk.internal.org.objectweb.asm.MethodWriter ([email protected])
53: 1799 71960 java.lang.invoke.MethodType ([email protected])
54: 1089 69696 java.net.URL ([email protected])
55: 121 50512 [Ljava.util.concurrent.ConcurrentHashMap$Node; ([email protected])
56: 3147 50352 jdk.internal.util.StrongReferenceKey ([email protected])
57: 1057 42280 java.io.ObjectStreamField ([email protected])
58: 1225 39200 java.io.File ([email protected])
59: 779 37392 jdk.internal.org.objectweb.asm.Frame ([email protected])
60: 243 25272 java.util.jar.JarFile$JarFileEntry ([email protected])
61: 794 25248 [Ljava.lang.String; ([email protected])
62: 622 24880 java.lang.NoSuchFieldException ([email protected])
63: 571 22840 java.util.LinkedHashMap$Entry ([email protected])
64: 474 22752 jdk.internal.ref.CleanerImpl$PhantomCleanableRef ([email protected])
65: 690 22080 jdk.internal.util.WeakReferenceKey ([email protected])
66: 828 19872 jdk.internal.org.objectweb.asm.ByteVector ([email protected])
67: 248 18848 [Ljava.lang.ref.SoftReference; ([email protected])
68: 118 17936 jdk.internal.org.objectweb.asm.ClassWriter ([email protected])
69: 380 16824 [Ljava.lang.invoke.LambdaForm$Name; ([email protected])
70: 625 15000 java.lang.Long ([email protected])
71: 463 14816 java.lang.invoke.LambdaForm$Name ([email protected])
72: 904 14464 java.lang.Object ([email protected])
73: 198 14256 java.lang.reflect.Constructor ([email protected])
74: 249 13944 java.util.zip.ZipFile$ZipFileInputStream ([email protected])
75: 334 13360 jdk.internal.org.objectweb.asm.Handler ([email protected])
76: 202 12928 java.util.concurrent.ConcurrentHashMap ([email protected])
77: 201 12864 jdk.internal.org.objectweb.asm.FieldWriter ([email protected])
78: 316 12640 java.util.WeakHashMap$Entry ([email protected])
79: 102 12240 java.io.ObjectStreamClass ([email protected])
80: 249 11952 java.util.zip.ZipFile$ZipFileInflaterInputStream ([email protected])
81: 359 11488 jdk.internal.org.objectweb.asm.Type ([email protected])
82: 465 11160 java.lang.invoke.ResolvedMethodName ([email protected])
83: 464 11136 jdk.internal.org.objectweb.asm.Edge ([email protected])
84: 341 10912 jdk.internal.math.FDBigInteger ([email protected])
85: 94 10728 [Ljava.lang.reflect.Field; ([email protected])
86: 266 10640 java.lang.NoSuchMethodException ([email protected])
87: 266 10640 java.security.CodeSource ([email protected])
88: 221 10608 java.lang.invoke.DirectMethodHandle$Constructor ([email protected])
89: 264 10560 sun.security.util.KnownOIDs ([email protected])
90: 75 10200 sun.nio.fs.UnixFileAttributes ([email protected])
91: 245 9800 java.lang.ref.SoftReference ([email protected])
92: 118 9440 jdk.internal.event.DeserializationEvent ([email protected])
93: 115 9200 [Ljava.util.WeakHashMap$Entry; ([email protected])
94: 368 8832 java.lang.module.ModuleDescriptor$Exports ([email protected])
95: 63 8384 [Ljava.lang.invoke.MethodHandle; ([email protected])
96: 146 8176 java.io.FileCleanable ([email protected])
97: 125 8000 java.lang.Class$ReflectionData ([email protected])
98: 323 7752 java.util.ImmutableCollections$Set12 ([email protected])
99: 121 7744 jdk.internal.org.objectweb.asm.SymbolTable ([email protected])
100: 70 7280 java.lang.invoke.InnerClassLambdaMetafactory ([email protected])
101: 144 6912 jdk.internal.org.objectweb.asm.AnnotationWriter ([email protected])
102: 167 6680 jdk.internal.loader.URLClassPath$JarLoader$2 ([email protected])
103: 199 6368 java.lang.invoke.MethodHandles$Lookup ([email protected])
104: 156 6240 java.util.StringJoiner ([email protected])
105: 153 6120 java.io.FileDescriptor ([email protected])
106: 126 6048 java.lang.invoke.LambdaForm ([email protected])
107: 77 6016 [Ljava.lang.reflect.Method; ([email protected])
108: 249 5976 java.util.zip.ZipFile$InflaterCleanupAction ([email protected])
109: 373 5968 java.lang.Byte ([email protected])
110: 74 5920 java.util.zip.ZipFile$Source ([email protected])
111: 82 5720 [Ljava.io.ObjectStreamField; ([email protected])
112: 40 5640 [Ljava.lang.ClassValue$Entry; ([email protected])
113: 234 5616 java.util.jar.Attributes$Name ([email protected])
114: 174 5568 java.util.concurrent.locks.ReentrantLock$NonfairSync ([email protected])
115: 98 5488 java.lang.Module ([email protected])
116: 219 5256 java.lang.PublicMethods$MethodList ([email protected])
117: 65 5200 java.net.URI ([email protected])
118: 215 5104 [Ljdk.internal.org.objectweb.asm.Type; ([email protected])
truncated...
Total 4452140 139359040
0.186 s/op
+totalAllocatedBytes: 139359040.000 bytes
+totalAllocatedInstances: 4452140.000 instances
+totalHeap: 521412608.000 bytes
Secondary result "dev.openfeature.sdk.benchmark.AllocationBenchmark.run:+totalAllocatedBytes":
139359040.000 bytes
Secondary result "dev.openfeature.sdk.benchmark.AllocationBenchmark.run:+totalAllocatedInstances":
4452140.000 instances
Secondary result "dev.openfeature.sdk.benchmark.AllocationBenchmark.run:+totalHeap":
521412608.000 bytes
# Run complete. Total time: 00:00:00
REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.
NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise
extra caution when trusting the results, look into the generated code to check the benchmark still
works, and factor in a small probability of new VM bugs. Additionally, while comparisons between
different JVMs are already problematic, the performance difference caused by different Blackhole
modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons.
Benchmark Mode Cnt Score Error Units
AllocationBenchmark.run ss 0.186 s/op
AllocationBenchmark.run:+totalAllocatedBytes ss 139359040.000 bytes
AllocationBenchmark.run:+totalAllocatedInstances ss 4452140.000 instances
AllocationBenchmark.run:+totalHeap ss 521412608.000 bytes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.280 s
[INFO] Finished at: 2024-10-23T12:37:24-04:00
[INFO] ------------------------------------------------------------------------