Tailscale incredibly slow, suspecting the add-on or HaOS to be the cause

[borgqueenx]

I apologise for opening this issue on the integration issues tracker, it should have been in the add-on of tailscale for HAOS.

When connected from anywhere but my own local lan network, the home assistant companion app is incredibly slow when using the tailscale network.
I have tested it using the openspeedtest addon for home assistant that tests speeds between devices, and i get a result of 0.6 till 0.7mbit download.
Weirdly, upload speed is not affected, and is fair across any test i have used.

Now i have tried a lot of things that makes me think there is something with either home assistant or the tailscale add-on.

Running my windows computer as exit node, which is connected to the same home router as the home assistant minipc, is very fast, i get 28mbit download. also when connecting to the windows computer through home assistant as exit node is fast.
I tried connecting to home assistant using the local lan address and tailscale IP, it doesn't matter for speeds.
Connection speeds towards the internet with home assistant as exit node is around double the download speed as connected to home assistant itself with openspeedtest (that tests speeds between devices)
i tried using the zerotier add-on on home assistant which works on normal speed as well.
cellular or other wifi networks do not matter for the speed test. only local lan is fast.
using the tailscale app on android, i note that in every time i test things, there is a direct connection to the device. DERP servers are (seemingly) not being used.
i also tried playing around with tailscale admin settings and add-on configuration without luck (enable/disable magicdns and some more)
Hardware on the home assistant is a Intel n100 cpu, with 12gb of ram. So should be capable enough for wireguard.

The issue started showing up at a seemingly random time and did not go away. i remember upgrading tailscale the same day the issue started and unfortunately did not go away. I didn't change any networking settings on any device on the day it started happening. the minipc runs HAOS directly. not virtualised or anything.

I also contacted tailscale support which cannot seem to find the cause either. a note they made though, is that zerotier uses TCP and tailscale uses UDP.

I also tried to report this issue on the HA forum, where someone else reported having the same issue, but no solutions were found.

I am a bit cautious sharing chunks of the log in public as i dont know what is personal identfiable info, i could send them by email or something or publicly here if you can say that the "netstack: could not connect to local backend server at [2600:1f18:xxxxxxxxxxxxxxxxxxxxxxx" things are not tied to anything personal?

As my desktop pc is fast with tailscale and zerotier is as well, i feel there is a issue with tailscale as a add-on, or something at least on home assistant that messes with tailscale as a add-on.

thanks for reading and hopefully this can be solved. 0.6mbit is unusable.

Thanks for reading!