Performance of Signals?

[rydmike]

Performance of Signals, was discussed before too and improved a lot. However, at least the Flutter part still seem a bit behind e.g. state_beacon and mobx. Or maybe the Rainbench benchmark is flawed or skewed in favor of it.

I just thought I would mention it, since I on request updated the benchmark and run the tests myself too and made a PR with my results and findings. See: https://github.com/jinyus/rainbench

Updated Rainbench test runs (Feb 3, 2024)

• Updated to use the latest version of each library
• Added Signals option to use Watch instead of context.watch.
• Fixed capability to actually select the option to use ValueNotifier with State Beacon.
• Tested in release mode on macOS (M1 Pro), with Flutter 3.16.9.
• The benchmark is restarted after each run to prevent GC interference.
• Used 20k raindrops and a bucket capacity of 200k for longer tests.

20k raindrops and a bucket capacity of 200k

Library	Raindrops/s	Time to fill bucket
1 state_beacon	20288	9.9s
2 mobx	12934	15.5s
3 state_beacon VN	9467	21.1s
4 value_LB (Throws exceptions after completion!!!)	9450	21.1s
5 solidart	9466	21.1s
6 signals Watch	8662	23.1s
7 signals context(watch)	7843	25.5s
8 context_watch	5504	36.3s
9 stream_builder	4549	44.0s

[rodydavis]

Ran the same tests with M3 Max 96GB ram.

One of the reasons this benchmark does not tell the whole story is just by changing the methods and rerunning you can get different results.

For example by using a callback vs builder with .watch(context) can make signals outperform state_beacon.

20k raindrops and a bucket capacity of 200k

• Signals: 17161.49 rps = 11.654s
• state_beacon: 14993.63 rps = 13.339

I think this benchmark is only one signal to look at. A more fair comparison would be deep computed chains and not just 1 layer deep

[rydmike]

For example by using a callback vs builder with .watch(context) can make signals outperform state_beacon.

Maybe so, but not sure I get what you mean with that in the context of this benchmark example. I'm pretty sure most Signals users are going to use the ways shown in the docs, meaning .watch(context) or very marginally faster Watch(), to trigger rebuilds of widgets.

As to the validity of this benchmark, I have not deep insights into that, I just found it interesting since it offered one point of comparison. It does looks valid, for the offered use case at least.

Maybe consider suggesting and submit improvements to the benchmark repo, that e.g. show how to optimize Signals better, if that is the case. Or maybe even provide some more complex use case example(s) as other benchmarks, with all optional solutions in this benchmark. Just a thought 😃

[jinyus]

I would like to see the code that's faster to confirm. I plan to do a more comprehensive test with deep vs wide dependency trees. I'm currently rewriting the core to be even more efficient so won't be able to get to that for now.

[rydmike]

That sounds very interesting.

I'm not sure if the diffs shown in above benchmark results have a major impact on any real world use case scenarios, maybe a bit in some cases, since the shown diffs are quite significant in the benchmark.

Other type of benchmarks and comparisons, like deep vs wide dependency trees sounds like an interesting study. At the very least an academic interest, but maybe more as, it may push for finding the reasons for any observed diffs, and thus maybe lead to improvements 👍🏻

Flutter and all signals?

While I love all the different signal options, I do think that for the FlutterDev community overall, it would probably be more beneficial if there was more of a joint effort to create and maintain "the best" signals dev experience for Dart and Flutter. Everything from perf, to API design, to features, docs, linter, dev tools, plus patterns and samples in docs.

Seems like you all @jinyus, @rodydavis, @nank1ro and @Solido would all have a lot more to offer that way. Plus it would be less of a maintenance burden for each of you going forward, when this becomes big, because it will. Just a wild thought and idea, I know it is very unlikely to happen! 😃

[Solido]

I hope this does not happen, at least to me, every implementations right now are very important in the exploration of api, conceptions and performances.

As much as I'm a Signals users, I'm really curious about all those bubbles and not sure what would be the best foundation for a common project. Even signals.dart seems so fresh and ... inconsistent in many parts. I would even say right now it's not for beginners because you need to tweak a lot.

While beacon or solid are more advanced in other areas, less in others and maybe more aligned with a philosophy.

Conclusion, let the chefs cook. All this require time and at least to me we're aiming for consolidation not adoption. Riverpod is still the recommandation for serious projets.