assert: improve partialDeepStrictEqual performance and add benchmark #56555

puskin94 · 2025-01-10T18:50:50Z

now that #54630 has landed, I took the liberty to review the code I wrote to look for improvements that would squeeze some more performance out of the comparison mechanism

simplified some logic
reviewed loops and introduced more breaking points to make them run for less cycles
added a benchmark file to test future possible regressions when it comes to partialDeepStrictEqual

Refs: #54630

nodejs-github-bot · 2025-01-10T18:50:56Z

Review requested:

@nodejs/performance

codecov · 2025-01-10T20:24:03Z

Codecov Report

Attention: Patch coverage is 57.74648% with 30 lines in your changes missing coverage. Please review.

Project coverage is 90.23%. Comparing base (6b3937a) to head (4502b76).
Report is 434 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/assert.js	57.74%	30 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #56555      +/-   ##
==========================================
+ Coverage   89.17%   90.23%   +1.05%     
==========================================
  Files         662      630      -32     
  Lines      191672   184968    -6704     
  Branches    36884    36192     -692     
==========================================
- Hits       170922   166904    -4018     
+ Misses      13614    11084    -2530     
+ Partials     7136     6980     -156

Files with missing lines	Coverage Δ
lib/assert.js	`96.50% <57.74%> (-2.54%)`	⬇️

... and 236 files with indirect coverage changes

lib/assert.js

puskin94 · 2025-01-26T20:11:33Z

Is anything else needed here?

lemire · 2025-01-26T20:32:40Z

@puskin94 Can you share the results of your benchmarking?

puskin94 · 2025-01-26T22:19:57Z

@puskin94 Can you share the results of your benchmarking?

of course 😄

➜  node git:(partial-deep-strict-equal-perf) ./node --no-warnings benchmark/assert/partial-deep-strict-equal.js 
assert/partial-deep-strict-equal.js datasetName="objects" size=1000 n=50: 169.87881572367604
assert/partial-deep-strict-equal.js datasetName="sets" size=1000 n=50: 141.24912248982653
assert/partial-deep-strict-equal.js datasetName="maps" size=1000 n=50: 151.36189759383777
assert/partial-deep-strict-equal.js datasetName="arrayBuffers" size=1000 n=50: 14.16071061277997
assert/partial-deep-strict-equal.js datasetName="dataViewArrayBuffers" size=1000 n=50: 16.398050206238278

aduh95 · 2025-01-27T10:38:59Z

lib/assert.js

+  // Create a map for faster lookups
+  const actualMap = new SafeMap();
+  const actualIterator = FunctionPrototypeCall(SafeSet.prototype[SymbolIterator], actual);


Why use a Map if we only store true in it?

Suggested change

// Create a map for faster lookups

const actualMap = new SafeMap();

const actualIterator = FunctionPrototypeCall(SafeSet.prototype[SymbolIterator], actual);

const actualSet = new SafeSet(

FunctionPrototypeCall(SafeSet.prototype[SymbolIterator], actual),

);

aduh95 · 2025-01-27T11:16:11Z

lib/assert.js


-  for (const expectedItem of safeExpected) {
-    // Check if the item is a zero or a -0, as these need to be handled separately
+  for (const expectedItem of new SafeArrayIterator(expected)) {


If we're looking for performance, we should probably be using a classic for(;;) loop. I'd recommend rolling back this change (i.e. put back the safeExpected assignment, because it has nothing to do with performance) for this PR, and make a new one switching to a for(;;) loop on which we can run benchmarks

if you do this tho you would have to call the ArrayFrom primordial. The performance enhancement was to adjust the "low hanging fruit" stuff I could get, "structure" wise, without influencing readability. IMHO the current implementation is better, but happy to change if you don't agree!

reverted the assignment tho

benchmark/assert/partial-deep-strict-equal.js

BridgeAR · 2025-01-28T22:24:37Z

lib/assert.js

+    if (isPrimitive(expectedItem)) {
+      if (actualSet.has(expectedItem)) {
+        actualSet.delete(expectedItem);


This could be improved further by just adding entries that are not detected. So we just lazily allocate the set for objects that are not reference identical on the other side.

This is how it's implemented in the current deepStrictEqual comparison.

puskin94 · 2025-02-07T15:34:06Z

Is anything else needed here?

puskin94 · 2025-02-21T14:01:19Z

is any change required here?

aduh95 · 2025-02-21T16:32:56Z

@BridgeAR can you clarify whether you're still blocking?

nodejs-github-bot · 2025-02-21T16:34:13Z

CI: https://ci.nodejs.org/job/node-test-pull-request/65363/

mcollina

lgtm

BridgeAR

Sorry for the late reply. I guess this improves a couple of cases, so that's great! :)
I believe there is more room for improvement by reusing the existing implementation. That could be used for a follow-up PR.
Before landing: could we run the benchmark to see the improvement as percentage? :)

lib/assert.js

Just did a new review

aduh95 · 2025-02-28T09:54:59Z

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1664/

Results

                                                                                      confidence improvement accuracy (*)   (**)  (***)
assert/partial-deep-strict-equal.js datasetName='arrayBuffers' size=1000 n=50                ***     -0.79 %       ±0.38% ±0.51% ±0.67%
assert/partial-deep-strict-equal.js datasetName='dataViewArrayBuffers' size=1000 n=50         **      0.64 %       ±0.45% ±0.60% ±0.78%
assert/partial-deep-strict-equal.js datasetName='maps' size=1000 n=50                        ***      0.56 %       ±0.26% ±0.35% ±0.46%
assert/partial-deep-strict-equal.js datasetName='objects' size=1000 n=50                     ***     -0.49 %       ±0.28% ±0.37% ±0.48%
assert/partial-deep-strict-equal.js datasetName='sets' size=1000 n=50                          *      0.34 %       ±0.26% ±0.35% ±0.45%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 5 comparisons, you can thus
expect the following amount of false-positive results:
  0.25 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.05 false positives, when considering a   1% risk acceptance (**, ***),
  0.01 false positives, when considering a 0.1% risk acceptance (***)

puskin94 · 2025-02-28T10:45:08Z

I must have messed up something with my local codebase. I just pushed again with new performance improvements, this should make some difference

lib/assert.js

aduh95 · 2025-02-28T10:57:04Z

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1665/

lib/assert.js

aduh95 · 2025-02-28T12:23:39Z

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1666/

BridgeAR · 2025-02-28T13:47:37Z

New benchmark

https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1667/

BridgeAR

If I am not mistaken, this mainly improves the performance for sets and maps that have mixed entries (primitives + objects). The current benchmark has no such entry.

I also guess that the primordial usage of the iterator is costly. Could you check if that's the case?

BridgeAR · 2025-03-03T18:39:12Z

lib/assert.js

+function isPrimitive(value) {
+  return typeof value !== 'object' || value === null;
+}


This should probably be inlined again. I guess that is where the performance degradation in the benchmark is coming from.

nodejs-github-bot added assert Issues and PRs related to the assert subsystem. needs-ci PRs that need a full CI run. labels Jan 10, 2025

BridgeAR previously requested changes Jan 13, 2025

View reviewed changes

lib/assert.js Outdated Show resolved Hide resolved

lib/assert.js Outdated Show resolved Hide resolved

lib/assert.js Outdated Show resolved Hide resolved

lib/assert.js Outdated Show resolved Hide resolved

lib/assert.js Outdated Show resolved Hide resolved

puskin94 force-pushed the partial-deep-strict-equal-perf branch from ac146c0 to 762399f Compare January 13, 2025 15:05

puskin94 requested a review from BridgeAR January 17, 2025 08:44

aduh95 reviewed Jan 27, 2025

View reviewed changes

puskin94 force-pushed the partial-deep-strict-equal-perf branch from 762399f to d061759 Compare January 27, 2025 11:08

aduh95 reviewed Jan 27, 2025

View reviewed changes

benchmark/assert/partial-deep-strict-equal.js Outdated Show resolved Hide resolved

puskin94 force-pushed the partial-deep-strict-equal-perf branch from d061759 to f1f220c Compare January 27, 2025 12:25

BridgeAR reviewed Jan 28, 2025

View reviewed changes

puskin94 force-pushed the partial-deep-strict-equal-perf branch from f1f220c to e932899 Compare January 29, 2025 08:00

aduh95 approved these changes Feb 21, 2025

View reviewed changes

aduh95 added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Feb 21, 2025

aduh95 requested a review from BridgeAR February 21, 2025 16:32

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 21, 2025

mcollina approved these changes Feb 23, 2025

View reviewed changes

BridgeAR reviewed Feb 27, 2025

View reviewed changes

lib/assert.js Outdated Show resolved Hide resolved

lib/assert.js Outdated Show resolved Hide resolved

puskin94 force-pushed the partial-deep-strict-equal-perf branch from e932899 to a07cf19 Compare February 28, 2025 10:43

BridgeAR reviewed Feb 28, 2025

View reviewed changes

lib/assert.js Outdated Show resolved Hide resolved

puskin94 force-pushed the partial-deep-strict-equal-perf branch from a07cf19 to b53b85c Compare February 28, 2025 11:21

BridgeAR reviewed Feb 28, 2025

View reviewed changes

lib/assert.js Outdated Show resolved Hide resolved

puskin94 force-pushed the partial-deep-strict-equal-perf branch from b53b85c to 989b529 Compare February 28, 2025 11:43

puskin94 force-pushed the partial-deep-strict-equal-perf branch from 989b529 to 2e4fbdb Compare February 28, 2025 13:27

BridgeAR approved these changes Feb 28, 2025

View reviewed changes

BridgeAR reviewed Mar 3, 2025

View reviewed changes

assert: improve partialDeepStrictEqual performance and add benchmark

4502b76

puskin94 force-pushed the partial-deep-strict-equal-perf branch from 2e4fbdb to 4502b76 Compare March 3, 2025 22:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

assert: improve partialDeepStrictEqual performance and add benchmark #56555

assert: improve partialDeepStrictEqual performance and add benchmark #56555

puskin94 commented Jan 10, 2025

nodejs-github-bot commented Jan 10, 2025

codecov bot commented Jan 10, 2025 •

edited

Loading

puskin94 commented Jan 26, 2025

lemire commented Jan 26, 2025

puskin94 commented Jan 26, 2025 •

edited

Loading

aduh95 Jan 27, 2025

aduh95 Jan 27, 2025

puskin94 Jan 27, 2025

BridgeAR Jan 28, 2025

puskin94 commented Feb 7, 2025

puskin94 commented Feb 21, 2025

aduh95 commented Feb 21, 2025

nodejs-github-bot commented Feb 21, 2025

mcollina left a comment

BridgeAR left a comment

aduh95 commented Feb 28, 2025 •

edited

Loading

puskin94 commented Feb 28, 2025

aduh95 commented Feb 28, 2025

aduh95 commented Feb 28, 2025

BridgeAR commented Feb 28, 2025

BridgeAR left a comment

BridgeAR Mar 3, 2025

assert: improve partialDeepStrictEqual performance and add benchmark #56555

Are you sure you want to change the base?

assert: improve partialDeepStrictEqual performance and add benchmark #56555

Conversation

puskin94 commented Jan 10, 2025

nodejs-github-bot commented Jan 10, 2025

codecov bot commented Jan 10, 2025 • edited Loading

Codecov Report

puskin94 commented Jan 26, 2025

lemire commented Jan 26, 2025

puskin94 commented Jan 26, 2025 • edited Loading

aduh95 Jan 27, 2025

Choose a reason for hiding this comment

aduh95 Jan 27, 2025

Choose a reason for hiding this comment

puskin94 Jan 27, 2025

Choose a reason for hiding this comment

BridgeAR Jan 28, 2025

Choose a reason for hiding this comment

puskin94 commented Feb 7, 2025

puskin94 commented Feb 21, 2025

aduh95 commented Feb 21, 2025

nodejs-github-bot commented Feb 21, 2025

mcollina left a comment

Choose a reason for hiding this comment

BridgeAR left a comment

Choose a reason for hiding this comment

aduh95 commented Feb 28, 2025 • edited Loading

puskin94 commented Feb 28, 2025

aduh95 commented Feb 28, 2025

aduh95 commented Feb 28, 2025

BridgeAR commented Feb 28, 2025

BridgeAR left a comment

Choose a reason for hiding this comment

BridgeAR Mar 3, 2025

Choose a reason for hiding this comment

codecov bot commented Jan 10, 2025 •

edited

Loading

puskin94 commented Jan 26, 2025 •

edited

Loading

aduh95 commented Feb 28, 2025 •

edited

Loading