Documentation about how the default dispatcher reuses connections

[brettwillis]

When developing a high-performance server, one of the first things I'd do in node-fetch and similar was make sure I used a new Agent({ keepAlive: true }) agent to make sure that I wasn't re-establishing and then immediately destroying fresh TCP connections for every request.

Now, particularly in the context of Node.js's built-in fetch(), my first question is: if and for how long are connections being kept alive and reused under the hood? Based on benchmarks, I'm guess it must be reusing connections in some way.

In a browser implementation of fetch(), this is purely up to the browser to optimise, and they are normally very good at pooling connections and even using pipelining and HTTP/2/3 etc.

In undici, these details are implemented by the dispatcher, and the in the case of build-in fetch() this is not something we can control or customise.

So my question/request is: can we have documentation that explains how the default dispatcher behaves in terms of reusing connections, opportunistically using pipelining and HTTP/2/3 etc.

Like I mentioned this is one of the first questions when developing a high-performance server, and currently we need to dig into the code to find there is a default dispatcher, and examine the implementation learn what it is doing.

[metcoder95]

Me again!

fetch uses the globalDispatcher set by undici. This dispatcher uses an Agent with its defaults. Under-the-hood, Agent creates, depending on the configuration passed, an instance of either a Pool or Client, if entirely based on the defaults, a Client is made (it depends on the number of connections passed).

Usually, this configuration works perfectly for many cases, but for more complex use cases, you can alter it by passing an Agent instance to the setGlobalDispatcher, another option is to do the same but pass a dispatcher-like object to fetch (usually a Pool or Client instance), e.g. fetch('url', { dispatcher }).

H2 support (H3 is not in Node.js yet) is experimental, and can be used by creating a client or agent with the flag allowH2: true.

Hope is pretty clear, but a PR is welcome to improve our documentation around those topics 🙂

[longzheng]

In case anyone else was looking, the default agent checks for connections === 1

undici/lib/dispatcher/agent.js

Lines 19 to 23 in e93dbd5

	function defaultFactory (origin, opts) {
	return opts && opts.connections === 1
	? new Client(origin, opts)
	: new Pool(origin, opts)
	}

[Galienna]

I'm also struggling with the way Undici's Agent is managing its connections. With Node's Agent, we had a clear documentation here, with keepAlive, maxSockets and especially scheduling.

Could you please explain what's the scheduling strategy here ? Is there a way ton configure it ?

[metcoder95]

When sending a request to mydns.com, do you first resolve the DNS then try to find a connection that matches the given IP ? Or do you directly try to find a connection that has been used for mydns.com before resolving the DNS ?

We leverage to node:dns#lookup for that as we use the socket defaults for IP resolution.
All pools are based on origins URLs, then whatever you've set as URL (either the direct IP or the DNS name), will be used to keep the Pool of connections for the downstream.

If the origin rotates IPs after a time, the connection will be dropped but a new one will be made automatically.

If seeking for a more customized behavior with DNS involvement, I'd recommend to use the dns interceptor, which caches and handles this automatically (but you can extend it if you need to).

[Galienna]

If the origin rotates IPs after a time, the connection will be dropped but a new one will be made automatically.

Maybe i missed something, but my tests show that undici's fetch & agent are reusing an old connection with an IP that no longer matches the requested URL. I don't see the old connection being dropped.

My test is the following :

// file test.js
// "undici": "6.21.0"
import undici from 'undici';

const testAgent = new undici.Agent({
    connections: 10,
    connect: {
        keepAlive: true,
        timeout: 5000
    }
});

async function sleep(ms) {
    return new Promise((resolve) => {
        setTimeout(resolve, ms);
    });
}

async function main() {
    while (1) {
        const response = await undici.fetch('https://some-partner.com', {
            dispatcher: testAgent,
            method: 'PATCH',
            body: '{"foo":"bar"}',
            headers: {'content-type': 'application/json'}
        });
        console.log('resp code '+response.statusText)
        
        await sleep(1000);
    }
}

main();

Then :

• In /etc/hosts, force the IP of some-partner.com to one of the IP provided by the GSLB : 10.117.30.10
• Observe connections with `watch -n 0.3 "netstat -anotv | grep -E 'Adresse|443'"
• Run the test with node v20.18.0 : node test.js
• Netstat shows a connection :

Proto Recv-Q Send-Q Adresse locale          Adresse distante        Etat       Timer
tcp        0      0 10.0.2.15:50376         10.117.30.10:443        ESTABLISHED keepalive (17,02/0/0)

• Edit /etc/hosts to simulate a rotation of IP. Set some-partner.com to : 10.117.28.10
• Netstat shows the same connection being reused over and over, even if i keep running the test for 5 min :

Proto Recv-Q Send-Q Adresse locale          Adresse distante        Etat       Timer
tcp        0      0 10.0.2.15:50376         10.117.30.10:443        ESTABLISHED keepalive (54,50/0/0)

If you add a log to see what dns.lookup is getting, you'll see it properly follows /etc/hosts

[metcoder95]

Sorry, I didn't explain myself correctly; it is automatically dropped as soon as the current connection (before rotation) is ended by the downstream or the client itself. As long as the IP stills reachable, undici will keep the connection open.

Once the connection is closed, undici will attempt to create a new connection, and do a new dns lookup. Depending on the OS settings and the DNS records cached, it will resolve to the new rotated IP.

If you are looking to manage these resolutions in a more preemptive way, again, the best is to use the dns interceptor to customize the way the records are cached and how often can be rotated.

[Galienna]

Thank you for the detailed answer :) I'll try to figure out what is dns interceptors and how to use it with fetch. Just to be sure, you're talking about this, right ?
NB : the search tool from the doc gives nothing when searching for "interceptor" ^^'

That being said, i don't understand why the default behaviour of undici is to ignore DNS TTL, especially after reading your comment on #3350. I'm confused

[metcoder95]

Thank you for the detailed answer :) I'll try to figure out what is dns interceptors and how to use it with fetch. Just to be sure, you're talking about this, right ?

Yes. And yeah, is quite hidden; sorry for that

That being said, i don't understand why the default behaviour of undici is to ignore DNS TTL, especially after reading your comment on #3350 (comment). I'm confused

It uses node:dns, so whatever the dns.lookup does, undici works with it. See dns implementation considerations for more info.