Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apparently we are sending way too many typing notifications #29107

Open
ara4n opened this issue Jan 27, 2025 · 2 comments
Open

Apparently we are sending way too many typing notifications #29107

ara4n opened this issue Jan 27, 2025 · 2 comments
Labels
S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect

Comments

@ara4n
Copy link
Member

ara4n commented Jan 27, 2025

Steps to reproduce

https://git.pixie.town/f0x/synapse-edulimit has been created as a c2s proxy which ratelimits typing notifications. However, the actual reason it's needed seems to be that EW sounds like it is sending way too many typing notif events. In theory, we should be sending an update when the user starts typing, and then a keepalive every ~30s, and/or a cancellation when they clear the composer.

But in practice

just the short bit of typing I did in this room, with 751 homeservers, would've sent 21028 HTTP requests so far

says https://matrix.to/#/!QQpfJfZvqxbCfeDgCj:matrix.org/$MRlYWUiNL0ib2gyPD40r16zVY1mxFDgR6Gm_5BeGams?via=matrix.org&via=envs.net&via=element.io

which sounds like 28 typing notifications just by sending 3 messages in a room.

f0x said:

⁦it seems very quick to detect that you stopped typing, which is then immediately followed by having to send another request because you started again⁦      ⠀

Which sounds like we might have a regression in the debouncing.

Outcome

What did you expect?

Typing notifs shouldn't DoS your server, let alone other servers.

What happened instead?

The act of typing a string of messages sent 50K HTTP hits in #twim. Which feels excessive.

Operating system

No response

Application version

Element Nightly version: 2025012301 Crypto version: Rust SDK 0.8.0 (21f7cc7), Vodozemac 0.8.1

How did you install the app?

No response

Homeserver

No response

Will you send logs?

No

@ara4n ara4n added the T-Defect label Jan 27, 2025
@dosubot dosubot bot added the S-Major Severely degrades major functionality or product features, with no satisfactory workaround label Jan 27, 2025
@t3chguy
Copy link
Member

t3chguy commented Jan 27, 2025

Screen.Recording.2025-01-27.at.11.32.25.mov

Half of the requests are CORS preflight which we cannot avoid. Looks like the current logic is to re-send every 10s rather than 30s and testing it, seems to behave exactly like that. With one caveat being if you empty the composer that sends a stopped typing signal. So if your typing style involves clearing the composer a lot then you would indeed see that pattern. This feels like something your HS should debounce over federation for risk of getting dev/null'd

@f0x52
Copy link

f0x52 commented Jan 27, 2025

Small correction: it's a c2s proxy, that filters typing notifs from local users, based on the amount of remote servers in the room they would be sent to.

Logs from yesterday, sending a couple of messages in TWIM. No clearing of the input field except for sending messages, though I do pause while typing and/or go back to reword stuff differently. It's hard to capture on video :')

16:22:52.721  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:23:02.228  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:23:15.301  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:23:25.303  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:02.155  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:04.465  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:18.630  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:18.860  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:19.058  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:29.122  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:29.914  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:39.976  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:40.120  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:50.177  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:26:54.811  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:04.868  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:04.908  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:14.964  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:14.991  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:25.107  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:30.429  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:40.432  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:44.530  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:54.588  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:27:54.669  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:28:04.724  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:28:20.248  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:28:26.110  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:31:11.351  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:31:21.349  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:31:21.355  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:31:31.412  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:31:36.682  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:31:46.683  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:31:47.182  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:31:57.241  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:31:58.725  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:32:08.784  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:32:18.011  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:32:28.013  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:32:30.725  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:32:36.221  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:34:55.678  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:34:56.770  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:35:07.670  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:35:12.912  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:36:00.904  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:36:10.906  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:36:13.371  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:36:23.430  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:36:23.871  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:36:33.928  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:36:35.365  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:36:45.421  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:36:50.031  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:00.128  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:00.165  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:10.222  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:10.592  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:20.650  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:23.722  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:33.779  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:33.786  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:43.845  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:44.972  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:55.031  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org
16:37:55.033  typing for 30000ms in !QQpfJfZvqxbCfeDgCj:matrix.org
16:38:05.037  stopped typing in !QQpfJfZvqxbCfeDgCj:matrix.org

You can see multiple events where it sends a 'stopped typing' 10 seconds after starting, immediately followed by a 'start typing', sent within the same second.
This is on element-web in firefox, but similar massive traffic spikes have happened from a different local user typing in element-desktop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect
Projects
None yet
Development

No branches or pull requests

3 participants