perf: improve equality comparison performance #173

Maksimka101 · 2023-12-11T15:14:40Z

Status

READY

Breaking Changes

NO

Description

I've noticed that the performance of comparing objects with collections can be significantly enhanced. These optimizations in my application led to a considerable increase in FPS, approximately by 3 times. Additionally, I've conducted tests to compare the performance between the old and new approaches

Todos

Tests
Documentation
Examples

Impact to Remaining Code Base

This PR will affect:

comparison logic

lib/src/equatable_comparison_utils.dart

felangel · 2023-12-11T15:21:55Z

Thanks for the PR! Are you able to share the benchmarks you used? I would love to run them locally and validate the results.

Maksimka101 · 2023-12-11T15:32:10Z

Thanks for the PR! Are you able to share the benchmarks you used? I would love to run them locally and validate the results.

Sure, here is the link: https://github.com/Maksimka101/equatable_benchmark

Maksimka101 · 2023-12-23T11:24:44Z

Hello

Any news?🙂

Do I need to fix something or provide more information?

felangel · 2024-01-02T19:10:10Z

Hello

Any news?🙂

Do I need to fix something or provide more information?

Apologies for the delay! I was slow to respond due to the holidays but I should have time to review this in the next day or two. Thanks again!

Maksimka101 · 2024-01-02T21:07:32Z

Thanks. No need to hurry. I forgot that normal people relax on holidays🙂. Happy new year

Maksimka101 · 2024-03-21T03:44:22Z

Hello

Can we merge this?

escamoteur · 2024-05-24T11:21:09Z

Just saw this. besides that I think it would definitely great to get this merged, why not borrow the optimization of fast_equatable and cache the hashcode if that really improves performance that much :-)

felangel · 2024-05-24T13:47:20Z

Just saw this. besides that I think it would definitely great to get this merged, why not borrow the optimization of fast_equatable and cache the hashcode if that really improves performance that much :-)

Will review this later today apologies that it fell through the cracks. Looks like ci was failing but just re-triggered ci to see the logs.

felangel · 2024-05-24T13:51:08Z

Looks like some tests are missing. I’ll take a look at the benchmarks and will see if I can get this merged later today. Apologies for the delay!

felangel · 2024-05-26T18:25:12Z

@Maksimka101 I've added benchmarks and run them against the implementation on master and am seeing the following results:

branch: master

EmptyEquatable
          total runs:  2 076 295   
          total time:     2.0000  s
         average run:          0 μs
         runs/second:   Infinity
               units:        100   
        units/second:   Infinity
       time per unit:     0.0000 μs

PrimitiveEquatable
          total runs:    810 588   
          total time:     2.0000  s
         average run:          2 μs
         runs/second:    500 000   
               units:        100   
        units/second: 50 000 000   
       time per unit:     0.0200 μs

CollectionEquatable (small)
          total runs:    443 978   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

CollectionEquatable (medium)
          total runs:    442 368   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

CollectionEquatable (large)
          total runs:    450 915   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

branch: feat/performance-improvement

EmptyEquatable
          total runs:  2 069 828   
          total time:     2.0000  s
         average run:          0 μs
         runs/second:   Infinity
               units:        100   
        units/second:   Infinity
       time per unit:     0.0000 μs

PrimitiveEquatable
          total runs:    823 014   
          total time:     2.0000  s
         average run:          2 μs
         runs/second:    500 000   
               units:        100   
        units/second: 50 000 000   
       time per unit:     0.0200 μs

CollectionEquatable (small)
          total runs:    490 253   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

CollectionEquatable (medium)
          total runs:    494 469   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

CollectionEquatable (large)
          total runs:    494 548   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

MacBook Pro (M1 Pro, 16GB RAM)
Dart SDK version: 3.3.4 (stable) (Tue Apr 16 19:56:12 2024 +0000) on "macos_arm64"

I'm not able to reproduce the significant performance increase you describe. Maybe DeepCollectionEquality has been optimized since you last tested?

Let me know if you're still able to reproduce the ~20% performance improvement, thanks!

felangel · 2024-05-26T18:33:40Z

Just saw this. besides that I think it would definitely great to get this merged, why not borrow the optimization of fast_equatable and cache the hashcode if that really improves performance that much :-)

Because the classes should be immutable (e.g. all fields should be final and have a const constructor). Classes that use package:fast_equatable are not immutable and cannot have a const constructor.

Maksimka101 · 2024-05-31T05:36:03Z

The thing is, our benchmarks differ. It's hard to notice, but in mine, all the collections are identical, while in yours, almost all of them differ. That's why mine gives such small values – it goes through all the fields, all the elements of the collection. Whereas in your benchmark, differences are found almost immediately, and the method completes quickly

Try to add this benchmark:

_runBenchmark(
  'CollectionEquatable (large) (all equal)',
  (index) => CollectionEquatable(
    list: List.generate(100, (i) => 1024),
    map: Map.fromEntries(
      // ignore: prefer_const_constructors
      List.generate(100, (i) => MapEntry('${1024}', 1024)),
    ),
    set: Set.from(List.generate(100, (i) => 1024)),
  ),
);

It'll give you the following results:

CollectionEquatable (large) (all equal)
          total runs:     20 718   
          total time:     2.0000  s
         average run:         96 μs
         runs/second:     10 417   
               units:        100   
        units/second:  1 041 667   
       time per unit:     0.9600 μs

In my app, one field in a huge store object was changing, and this happened very frequently. My case is not the most common, but it's also not rare. The comparison operation should be fast not only for different values but also for identical ones :)

felangel · 2024-06-04T05:02:01Z

The thing is, our benchmarks differ. It's hard to notice, but in mine, all the collections are identical, while in yours, almost all of them differ. That's why mine gives such small values – it goes through all the fields, all the elements of the collection. Whereas in your benchmark, differences are found almost immediately, and the method completes quickly

Try to add this benchmark:
_runBenchmark(
  'CollectionEquatable (large) (all equal)',
  (index) => CollectionEquatable(
    list: List.generate(100, (i) => 1024),
    map: Map.fromEntries(
      // ignore: prefer_const_constructors
      List.generate(100, (i) => MapEntry('${1024}', 1024)),
    ),
    set: Set.from(List.generate(100, (i) => 1024)),
  ),
);
It'll give you the following results:
CollectionEquatable (large) (all equal)
          total runs:     20 718   
          total time:     2.0000  s
         average run:         96 μs
         runs/second:     10 417   
               units:        100   
        units/second:  1 041 667   
       time per unit:     0.9600 μs
In my app, one field in a huge store object was changing, and this happened very frequently. My case is not the most common, but it's also not rare. The comparison operation should be fast not only for different values but also for identical ones :)

Thanks for the reply! I updated the benchmarks and am able to reproduce the slowdown in larger static datasets. Will look at it a bit more closely in the next few days. I want to take a closer look at why DeepCollectionEquality is suboptimal and ideally open a PR to improve the performance in package collection so that more packages benefit from it.

Maksimka101 · 2024-06-05T12:42:10Z

I've researched it a bit and I think the DeepCollectionEqality is slow by design due to is flexible API. It creates a map to compare sets and probably maps. Also, it doesn't use the == operator directly so it has some overhead

But hope I'm wrong :)

felangel · 2024-10-11T05:33:57Z

Planning to land this ASAP but it is still missing quite a few tests for the various equatable_utils APIs (e.g iterableEquals, objectEquals, setEquals, etc.). I already found several bugs in the original implementations and wouldn't be surprised if there were still some lingering bugs in the current version of this PR.

felangel · 2024-10-11T05:36:29Z

lib/src/equatable_utils.dart

-    final unitA = a[i];
-    final unitB = b[i];
-    if (_isEquatable(unitA) && _isEquatable(unitB)) {
-      return unitA == unitB;


This implementation is incorrect since this will incorrectly return true when the first two objects are equal in a list of props (even if the rest aren't):

class Person with EquatableMixin { Person({required this.name}); final String name; @override List<Object?> get props => [name]; } test('...', () { final alice = Person(name: 'Alice'); expect(equals([alice, null], [alice, -1]), isFalse); });

The above test incorrectly fails.

felangel · 2024-10-11T05:37:02Z

lib/src/equatable_utils.dart

-    if (_isEquatable(unitA) && _isEquatable(unitB)) {
-      return unitA == unitB;
-    } else if (unitA is Set && unitB is Set) {
-      return _setEquals(unitA, unitB);


Same goes for the rest of these -- we can't just return true early without actually checking each unit in the list.

Maksimka101 · 2024-10-11T16:12:50Z

You are right. I found one bug in the setEquals function

Also, I've added the @pragma('vm:prefer-inline') annotatio to the listEquals function which improved performance by 20%-10% for some cases:

Benchmark details (when compiled to the exe file)

With the inline pragma

EmptyEquatable
          total runs:  1 598 292   
          total time:     2.0000  s
         average run:          1 μs
         runs/second:  1 000 000   
               units:        100   
        units/second: 100 000 000   
       time per unit:     0.0100 μs

PrimitiveEquatable
          total runs:    678 313   
          total time:     2.0000  s
         average run:          2 μs
         runs/second:    500 000   
               units:        100   
        units/second: 50 000 000   
       time per unit:     0.0200 μs

CollectionEquatable (static, small)
          total runs:    139 323   
          total time:     2.0000  s
         average run:         14 μs
         runs/second:     71 429   
               units:        100   
        units/second:  7 142 857   
       time per unit:     0.1400 μs

CollectionEquatable (static, medium)
          total runs:    101 675   
          total time:     2.0000  s
         average run:         19 μs
         runs/second:     52 632   
               units:        100   
        units/second:  5 263 158   
       time per unit:     0.1900 μs

CollectionEquatable (static, large)
          total runs:     33 898   
          total time:     2.0000  s
         average run:         59 μs
         runs/second:     16 949   
               units:        100   
        units/second:  1 694 915   
       time per unit:     0.5900 μs

CollectionEquatable (dynamic, small)
          total runs:    458 845   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

CollectionEquatable (dynamic, medium)
          total runs:    460 127   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

CollectionEquatable (dynamic, large)
          total runs:    465 185   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

Without the inline pragma

EmptyEquatable
          total runs:  1 332 271   
          total time:     2.0000  s
         average run:          1 μs
         runs/second:  1 000 000   
               units:        100   
        units/second: 100 000 000   
       time per unit:     0.0100 μs

PrimitiveEquatable
          total runs:    585 970   
          total time:     2.0000  s
         average run:          3 μs
         runs/second:    333 333   
               units:        100   
        units/second: 33 333 333   
       time per unit:     0.0300 μs

CollectionEquatable (static, small)
          total runs:    133 655   
          total time:     2.0000  s
         average run:         14 μs
         runs/second:     71 429   
               units:        100   
        units/second:  7 142 857   
       time per unit:     0.1400 μs

CollectionEquatable (static, medium)
          total runs:     99 270   
          total time:     2.0000  s
         average run:         20 μs
         runs/second:     50 000   
               units:        100   
        units/second:  5 000 000   
       time per unit:     0.2000 μs

CollectionEquatable (static, large)
          total runs:     33 109   
          total time:     2.0001  s
         average run:         60 μs
         runs/second:     16 667   
               units:        100   
        units/second:  1 666 667   
       time per unit:     0.6000 μs

CollectionEquatable (dynamic, small)
          total runs:    423 572   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

CollectionEquatable (dynamic, medium)
          total runs:    410 959   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

CollectionEquatable (dynamic, large)
          total runs:    417 545   
          total time:     2.0000  s
         average run:          4 μs
         runs/second:    250 000   
               units:        100   
        units/second: 25 000 000   
       time per unit:     0.0400 μs

felangel · 2024-10-11T19:54:53Z

lib/src/equatable_utils.dart

 bool iterableEquals(Iterable<Object?> a, Iterable<Object?> b) {
+  assert(


Thoughts on calling setEquals instead of asserting here?

if (a is Set && b is Set) return setEquals(a, b);

I don't expect anyone to use this function. I'm not sure it's even exported. So I'd prefer to leave it simple, without extra ifs

felangel · 2024-10-16T00:44:45Z

@Maksimka101 with the latest changes, the benchmarks indicate the performance is worse than the original (in which we used DeepCollectionEquality) so I'm not sure there's any reason to proceed. FYI, I'm just comparing the benchmark results via dart run (JIT)

Maksimka101 · 2024-10-16T06:17:46Z

You're right, but only partially. In JIT mode, the current version is indeed faster in 5 out of 8 tests. But in AOT mode, the new implementation is significantly faster in 7 out of 8 tests. And this is very important because most Dart programs are compiled AOT

I'll try to figure out why it performs so poorly in JIT, and I'll also look at the difference when compiling to JS

Edit: New implementation is significantly faster in 6 of 8 tests and slightly slower in 2 of 8 tests when compiled to JS. (compiled with the -O2 flag, launched on node v22.4.0)

…rformance

Maksimka101 · 2024-10-16T11:01:38Z

@felangel I improved performance in almost every test across all build options. The largest performance drop is 6.3% in JS for the PrimitiveEquatable benchmark. The biggest performance boost is also in JS, in CollectionEquatable (static, medium), with a 356% increase :)

In the end, I just marked 2 functions for inline, but I ran so many experiments...

By the way, you can check out the performance comparison charts here: https://docs.google.com/spreadsheets/d/1e5g_URJ6oFc76e-YYhDXVBeqqzV7ZN0vDuh-3nPxGGY/edit?usp=sharing

felangel · 2024-10-16T13:32:35Z

@felangel I improved performance in almost every test across all build options. The largest performance drop is 6.3% in JS for the PrimitiveEquatable benchmark. The biggest performance boost is also in JS, in CollectionEquatable (static, medium), with a 356% increase :)

In the end, I just marked 2 functions for inline, but I ran so many experiments...

By the way, you can check out the performance comparison charts here: https://docs.google.com/spreadsheets/d/1e5g_URJ6oFc76e-YYhDXVBeqqzV7ZN0vDuh-3nPxGGY/edit?usp=sharing

Thanks! I’ll take a closer look later today 💙

felangel · 2024-11-20T18:08:23Z

Will update the benchmarks and include both AOT and JIT versions and plan to merge and publish this later today. Sorry for the delay and thanks for all your contributions and time 💙

felangel

LGTM thanks so much for the energy and time you put into this -- I really appreciate it! 💙

Maksimka101 added 2 commits December 11, 2023 15:21

Feat: improve comparison performance

991cebf

Feat: optimize non collection properties comparison

6204491

Maksimka101 requested a review from felangel as a code owner December 11, 2023 15:14

felangel reviewed Dec 11, 2023

View reviewed changes

lib/src/equatable_comparison_utils.dart Outdated Show resolved Hide resolved

Refactor: move comparison logic back to the equatable_utils.dart file

45a0377

Maksimka101 added 3 commits December 11, 2023 18:38

Fix: sets comparison

88052e1

Style: reformat code

312a2a8

Fix: sets comparison in the _objectsEquals function

c3f4aef

felangel closed this May 24, 2024

felangel reopened this May 24, 2024

felangel assigned Maksimka101 May 24, 2024

felangel added the enhancement New feature or request label May 24, 2024

refactor: minor updates to code consistency

725f521

felangel added 2 commits October 10, 2024 23:37

Merge branch 'master' into feat/performance-improvement

b21514f

refactor: simplify equals implementation and fix coverage

8d283b0

felangel changed the title ~~Feat/performance improvement~~ perf: improve equality comparison performance Oct 11, 2024

felangel added 3 commits October 11, 2024 00:16

chore: update benchmarks

6ab28df

chore: adjust comments

fed7e05

refactor: simplify equatable utils

3f5afff

felangel reviewed Oct 11, 2024

View reviewed changes

Fix the setEquals function and write tests for the *Equals functions

261a28b

felangel reviewed Oct 11, 2024

View reviewed changes

Inline the _isEquatable and objectsEquals functions to improve pe…

751a2b0

…rformance

Merge branch 'master' into feat/performance-improvement

29e6c77

felangel added 2 commits November 20, 2024 21:42

chore: update benchmark results

bed68af

chore: minor adjustments

9e0983b

felangel approved these changes Nov 21, 2024

View reviewed changes

felangel merged commit a679c3d into felangel:master Nov 21, 2024
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: improve equality comparison performance #173

perf: improve equality comparison performance #173

Maksimka101 commented Dec 11, 2023

felangel commented Dec 11, 2023

Maksimka101 commented Dec 11, 2023 •

edited

Loading

Maksimka101 commented Dec 23, 2023

felangel commented Jan 2, 2024

Maksimka101 commented Jan 2, 2024

Maksimka101 commented Mar 21, 2024

escamoteur commented May 24, 2024

felangel commented May 24, 2024 •

edited

Loading

felangel commented May 24, 2024

felangel commented May 26, 2024 •

edited

Loading

felangel commented May 26, 2024

Maksimka101 commented May 31, 2024

felangel commented Jun 4, 2024

Maksimka101 commented Jun 5, 2024

felangel commented Oct 11, 2024

felangel Oct 11, 2024

felangel Oct 11, 2024

Maksimka101 commented Oct 11, 2024

With the inline pragma

Without the inline pragma

felangel Oct 11, 2024

Maksimka101 Oct 11, 2024

felangel commented Oct 16, 2024 •

edited

Loading

Maksimka101 commented Oct 16, 2024 •

edited

Loading

Maksimka101 commented Oct 16, 2024

felangel commented Oct 16, 2024

felangel commented Nov 20, 2024

felangel left a comment

		bool iterableEquals(Iterable<Object?> a, Iterable<Object?> b) {
		assert(

perf: improve equality comparison performance #173

perf: improve equality comparison performance #173

Conversation

Maksimka101 commented Dec 11, 2023

Status

Breaking Changes

Description

Todos

Impact to Remaining Code Base

felangel commented Dec 11, 2023

Maksimka101 commented Dec 11, 2023 • edited Loading

Maksimka101 commented Dec 23, 2023

felangel commented Jan 2, 2024

Maksimka101 commented Jan 2, 2024

Maksimka101 commented Mar 21, 2024

escamoteur commented May 24, 2024

felangel commented May 24, 2024 • edited Loading

felangel commented May 24, 2024

felangel commented May 26, 2024 • edited Loading

felangel commented May 26, 2024

Maksimka101 commented May 31, 2024

felangel commented Jun 4, 2024

Maksimka101 commented Jun 5, 2024

felangel commented Oct 11, 2024

felangel Oct 11, 2024

Choose a reason for hiding this comment

felangel Oct 11, 2024

Choose a reason for hiding this comment

Maksimka101 commented Oct 11, 2024

With the inline pragma

Without the inline pragma

felangel Oct 11, 2024

Choose a reason for hiding this comment

Maksimka101 Oct 11, 2024

Choose a reason for hiding this comment

felangel commented Oct 16, 2024 • edited Loading

Maksimka101 commented Oct 16, 2024 • edited Loading

Maksimka101 commented Oct 16, 2024

felangel commented Oct 16, 2024

felangel commented Nov 20, 2024

felangel left a comment

Choose a reason for hiding this comment

Maksimka101 commented Dec 11, 2023 •

edited

Loading

felangel commented May 24, 2024 •

edited

Loading

felangel commented May 26, 2024 •

edited

Loading

felangel commented Oct 16, 2024 •

edited

Loading

Maksimka101 commented Oct 16, 2024 •

edited

Loading