I undertook the project of interpreting and implementing the IETF draft QUIC-FEC1 in April 2024 to fulfill a graduation requirement with three gracious PhD students acting as my advisors. I subsequently labored on it for the next 5 months in parallel to other classes and obligations. It was an arduous process to read RFC 9000 and source dive quic-go to find the places where I needed to make the necessary changes. The descriptions within the draft were also high-level, which meant I was making all the design decisions with respect to data structures and new interfaces. The following text contains slightly-modified passages from the report I wrote after finishing my edits and running throughput tests in case you'd like a bit more context with respect to what I actually did. Also, the TLDR of my results is that adding forward erasure correction is feasible, but it's extremely difficult to do and really only thrives in high loss and latency networks. This is not the norm, which means this technology is unnecessary in most cases. Thanks for reading :)
QUIC2 is a new general-purpose transport protocol, first standardized by the Internet Engineering Task Force (IETF) in 2021, which combines features from mature protocols such as TCP3, UDP4, and TLS5. QUIC differentiates itself by offering secure connections by default, promises fewer round trips before data is transmitted, fixes the head-of-line blocking present with TCP, and is made to support a broad array of transport services. Hypertext Transfer Protocol Version 3.0 (HTTP/3)6, which is the newest version of the Hypertext Transfer Protocol (HTTP) and is the most popular application-level protocol built upon QUIC, is now used by 8.2% of all websites7. As QUIC increases in usage, it is critical to find ways in order to improve its performance amongst different classes of applications. IFC services, which see use within the context of airplane Wi-Fi by means of satellite or base station connectivity, offer a unique value proposition for QUIC due to their steady growth8, high-delays, and lossy nature9. A consequence of such a network is many retransmissions, which is the default loss-recovery mechanism of QUIC and includes the downside of a degraded user experience. However, by taking advantage of QUIC’s extensibility, it is possible to introduce new types of loss recovery mechanisms, which may better suit such networks. FEC is one such mechanism, which trades bandwidth for redundancy in the attempt to reduce the need for retransmissions, thereby increasing the quality of the connection. In this project, we investigate the performance of FEC relative to the pre-existing retransmission-only based loss recovery mechanism by extending the quic-go repository10 and running simulated network tests with conditions that mimic IFC networks.
For our experiments, we used a client-server file-download scenario. On each interface, we used netem to control the bandwidth, loss rate, and latency with values of 50Mbit/s, .1%, and 50ms, respectively. These values were chosen due to being within the range of Starlink network conditions as shown by Mohan et al.11. Each test was repeated 50 times. Seen in the figures below are the file download times from 1MB, 10MB, and 20MB files, respectively. The trend derived from these figures is that the larger the file size, the larger the discrepancy is between the median times of retransmission-only QUIC and FEC-based QUIC. This result can largely be attributed to the fact that in addition to the 1.5x overhead caused from the two-thirds coding rate, a large amount of spurious retransmissions occur. This is because with a packet-based loss detection threshold of 3 packets as defined in the QUIC loss recovery document12, spurious retransmission may commonly occur as the packet has been recovered too late. Consequently, this partly explains why the 10MB example has less of a discrepancy between FEC and non-FEC versions as there is simply less time for spurious retransmissions to occur. Another reason may also be because algorithms like slow-start play outsized roles during very short connections and overshadow the impacts of FEC. Although disappointing, these results are consistent with those from François Michel as seen in the figure below. Using network conditions derived from Rula et al.9, Michel also showed how small file sizes showed less of a median DCT discrepancy than with larger files.
The central entry point is the quic.Transport. A transport manages QUIC connections running on a single UDP socket. Since QUIC uses Connection IDs, it can demultiplex a listener (accepting incoming connections) and an arbitrary number of outgoing QUIC connections on the same UDP socket.
udpConn, err := net.ListenUDP("udp4", &net.UDPAddr{Port: 1234})
// ... error handling
tr := quic.Transport{
Conn: udpConn,
}
ln, err := tr.Listen(tlsConf, quicConf)
// ... error handling
go func() {
for {
conn, err := ln.Accept()
// ... error handling
// handle the connection, usually in a new Go routine
}
}()The listener ln can now be used to accept incoming QUIC connections by (repeatedly) calling the Accept method (see below for more information on the quic.Connection).
As a shortcut, quic.Listen and quic.ListenAddr can be used without explicitly initializing a quic.Transport:
ln, err := quic.Listen(udpConn, tlsConf, quicConf)
When using the shortcut, it's not possible to reuse the same UDP socket for outgoing connections.
As mentioned above, multiple outgoing connections can share a single UDP socket, since QUIC uses Connection IDs to demultiplex connections.
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) // 3s handshake timeout
defer cancel()
conn, err := tr.Dial(ctx, <server address>, <tls.Config>, <quic.Config>)
// ... error handlingAs a shortcut, quic.Dial and quic.DialAddr can be used without explictly initializing a quic.Transport:
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) // 3s handshake timeout
defer cancel()
conn, err := quic.Dial(ctx, conn, <server address>, <tls.Config>, <quic.Config>)Just as we saw before when used a similar shortcut to run a server, it's also not possible to reuse the same UDP socket for other outgoing connections, or to listen for incoming connections.
QUIC is a stream-multiplexed transport. A quic.Connection fundamentally differs from the net.Conn and the net.PacketConn interface defined in the standard library. Data is sent and received on (unidirectional and bidirectional) streams (and, if supported, in datagrams), not on the connection itself. The stream state machine is described in detail in Section 3 of RFC 9000.
Note: A unidirectional stream is a stream that the initiator can only write to (quic.SendStream), and the receiver can only read from (quic.ReceiveStream). A bidirectional stream (quic.Stream) allows reading from and writing to for both sides.
On the receiver side, streams are accepted using the AcceptStream (for bidirectional) and AcceptUniStream functions. For most user cases, it makes sense to call these functions in a loop:
for {
str, err := conn.AcceptStream(context.Background()) // for bidirectional streams
// ... error handling
// handle the stream, usually in a new Go routine
}These functions return an error when the underlying QUIC connection is closed.
There are two slightly different ways to open streams, one synchronous and one (potentially) asynchronous. This API is necessary since the receiver grants us a certain number of streams that we're allowed to open. It may grant us additional streams later on (typically when existing streams are closed), but it means that at the time we want to open a new stream, we might not be able to do so.
Using the synchronous method OpenStreamSync for bidirectional streams, and OpenUniStreamSync for unidirectional streams, an application can block until the peer allows opening additional streams. In case that we're allowed to open a new stream, these methods return right away:
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
str, err := conn.OpenStreamSync(ctx) // wait up to 5s to open a new bidirectional streamThe asynchronous version never blocks. If it's currently not possible to open a new stream, it returns a net.Error timeout error:
str, err := conn.OpenStream()
if nerr, ok := err.(net.Error); ok && nerr.Timeout() {
// It's currently not possible to open another stream,
// but it might be possible later, once the peer allowed us to do so.
}These functions return an error when the underlying QUIC connection is closed.
The quic.Config struct passed to both the listen and dial calls (see above) contains a wide range of configuration options for QUIC connections, incl. the ability to fine-tune flow control limits, the number of streams that the peer is allowed to open concurrently, keep-alives, idle timeouts, and many more. Please refer to the documentation for the quic.Config for details.
The quic.Transport contains a few configuration options that don't apply to any single QUIC connection, but to all connections handled by that transport. It is highly recommend to set the StatelessResetToken, which allows endpoints to quickly recover from crashes / reboots of our node (see Section 10.3 of RFC 9000).
See the example server. Starting a QUIC server is very similar to the standard library http package in Go:
http.Handle("/", http.FileServer(http.Dir(wwwDir)))
http3.ListenAndServeQUIC("localhost:4242", "/path/to/cert/chain.pem", "/path/to/privkey.pem", nil)See the example client. Use a http3.RoundTripper as a Transport in a http.Client.
http.Client{
Transport: &http3.RoundTripper{},
}