-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[aeron] introduce integration #1048
Conversation
result match | ||
case Present((tag2, messages)) => | ||
// verify message type matches expected type | ||
if tag2 != tag.toString then |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should ideally find a way to disallow toString
on opaque types... can we use raw instead here?
* | ||
* Messages must have a ReadWriter instance to be published or consumed. | ||
*/ | ||
type AsMessage[A] = ReadWriter[A] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reason for a type alias instead of opaque type?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My initial attempt was an opaque type but upickle's macros don't like it. I also had tried fury but had issues with encoders never finishing to compile (it uses runtime code generation)
val backpressured = Abort.fail(Backpressured()) | ||
|
||
// temporary storage for reassembled message | ||
var result: Maybe[(String, Chunk[A])] = Absent |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this safe to be used with Choice? Seems highly uncommon combination...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the code here suspends all side effects with IO
. The issue with Choice is related to mutability across multiple computation steps without IO
suspensions.
Co-authored-by: Adam Hearn <[email protected]>
Co-authored-by: Adam Hearn <[email protected]>
@hearnadam can we merge this? |
Co-authored-by: Adam Hearn <[email protected]>
Problem
High-performance solutions often require using inter-process communication (IPC) for extremely low latencies. It essentially uses in-memory communication between different processes, without even touching the network stack. Another common technique for optimization is using UDP instead of TCP for remote communication. TCP has a series of mitigations for unreliable transport channels that aren't necessary in more stable topologies like data centers. Aeron provides more lightweight mechanisms for reliability on top of UDP.
Solution
This PR integrates with Aeron to provide both IPC and UDP communication with a simple API based on
Stream
. Please see the scaladocs for more information.Notes
FragmentAssembler
can handle it. I'll follow up on this.Checklist