Support Futures / Asynchronous Results

[shanielh]

No description provided.

[malcolmredheron]

I just found to my surprise that Cask doesn't seem to have any support for returning a Future of a response instead of a response. TBH, I'm shocked. Is the idea that I block the thread until I have my response ready? Or am I missing something? I can't see anything about this in the docs

[lihaoyi]

@malcolmredheron Yes blocking a thread is generally fine. What's your RPS and number of concurrent requests that this is causing issues for you?

[malcolmredheron]

Well, we aren't in production yet so right now anything would be fine. We have a server that uses akka streams and actors and I hate the result. While converting it to cask and castor I noticed that we can't return a future and got very confused

Everything else of yours that I have seen has been so thoughtfully designed -- is there a principle behind this decision? I understand that I could tie up a thread while I talk to my backends, but I have future-based apis for those things and Scala makes futures pretty easy. Why do you prefer blocking in this case?

[lihaoyi]

Yes, the principle is that when discussing performance you need profiles and benchmarks. If you do not know your performance bottlenecks, performance requirements, or performance limits, worrying about tying up threads is a 100% waste of time. Just spawn more threads, and when that starts causing problems come back and we can have a more productive discussion on how your service can be optimized

[malcolmredheron]

I totally agree about the important of profiling and tuning real systems, not imaginary ones. And I want to use Cask. So far it's been clean and simple

At the same time, I'm trying to make decisions that are at least roughly setting myself up for future success. A thread, with its own stack, takes much more memory than a future. Has that increased memory usage never been a problem in your experience?

[lihaoyi]

It has not caused problems in my experience, but everyone's circumstances are different, so it's hard to generalize. e.g. my work has largely been in B2B and internal-facing systems, which have very different performance characteristics than e.g. consumer-facing web, ad-tech, and other vertical industries,

What I can generalize is that for the longest time, Youtube and Dropbox were mostly written in Python, Stripe and Github and Shopify are in Ruby, and Facebook was in vanilla PHP: all simple blocking tech stacks in addition to being 20-40x slower than JVM. If you think you're going to have more traffic than Youtube/Shopify/Facebook then maybe you need to worry about asynchrony and threading overhead, but for the bulk of folks who aren't going to be the next Youtube/Shopify/Facebook worrying about thread blocking overhead is a complete waste of time

[He-Pin]

I think Lihaoyi is right here, at least at Alibaba, where we mostly write blocking code. In a recent project, where we are using Netty, fastparse, and Pekko as a gateway to translate Taobao to Taobao English, the code is all async. Still, it is, really, very hard to reason about. Yes, performance is good, but it is not easy to maintain by others.

I think we can use Virtual threads, where your blocking code runs in a virtual thread. At least, it will not be a problem if you migrate to JDK 21/24 if your company has a JDK team that backports JEP 491 to JDK 21.

[malcolmredheron]

Ok. Thanks everyone. I'll set my fears aside for now :-)

@He-Pin, one question, though: was your hard-to-maintain code using futures or callbacks? I have so much future-oriented code in the browser already that I assume anyone on my team will be familiar with it

[He-Pin]

@malcolmredheron no, at least one promise is a single queue, order is a pain

[He-Pin]

@shanielh hi, the virtual thread support is in 0.9.6, which is quite simple, I think you can take a look at it.