Ensure e2e messages trigger unread counts and push notifs.

[ara4n]

No description provided.

[ara4n]

Have put a suggested PR at matrix-org/synapse#1131

[ara4n]

It will presumably need clientside support as well to let clients override these new base rules, however.

Alternatively, @dbkr, should this be being done entirely with clientside rules in the first place (with no defaults from the server)?

[richvdh]

Currently the badge counts are calculated by the server, because calculating them correctly requires having scrollback all the way back to the read-marker.

Solutions might be:

• make sure that the client has lots of scrollback (possibly all of it, in order to implement client-side search), or
• put mentions in metadata, per Displaynames inserted by tab completion should include some kind of metadata to link them to userids #553, and then send it in the clear, so that the server can manage it; however, this would only solve the problem for mentions, and not custom keywords.

[richvdh]

Initial suggestion is that we would like to avoid putting mentions in cleartext, and instead have the client back-paginate at least as far as the read-receipt in order to calculate badge counts on the client side.

[richvdh]

We had a longer discussion about this.

There are a number of options:

1. Revisit the way notifications work. Perhaps we don't actually need to support mentions and keywords in e2e rooms - is it sufficient to just get a notification on each event?

   Maybe... my problem is partly that the internal rooms I care about (E2E internal, particularly) end up somewhat indistinguishable from the high-traffic public rooms (Matrix HQ); a "high-priority" flag which makes the room turn red as if there was a mention might help me here. Still, I think really I want to get a special notification when someone mentions me.

2. Have the sender send a list of users to bing in a separate field in the event, and send it in the clear.

   The disadvantages are (a) a privacy leak, and (b) it won't help with custom keywords.

   The privacy leak is mitigated by the fact that the room membership list is already in the clear, as are read-receipts and event senders (you can infer pretty strongly who was mentioned in a message by who is first to read and reply to it). We could also make the choice between leaking mentions and working notifications a user choice via a "security level" slider.

   We might also need to consider what UX could make it clear that custom keywords won't work in E2E rooms.

3. Similar to 2, but have the sender send the recipient a "notification" in a to-device event instead.

   This might help mitigate the privacy leak, except not really because the participating HSes can still see the to-device events being sent. It would also add even more pain to the process of sending an e2e event, which is painful enough already.

4. Implement all notifications and badge counts on the client side, by having clients back-paginate up to the read-receipt.

   This would be the only way to get notifications to work properly for custom keywords as well as mentions. It would, however, be quite difficult to implement, as it doesn't fit well with the js-sdk's current model. We would need to back-paginate each room in the background looking for keywords; meanwhile events would continue to arrive, which would be tricky to handle without races. The user might read a room on another device, so every time a self-read-receipt arrived, we would need to restart this process.

   This might also cause a significant battery drain on mobile devices, but we're already effectively doing this on android and it doesn't seem to be a particular problem for me.

The viable options seem to be 2 and 4. Neither is ideal. One thing that did become obvious during discussion was that 4 (do notifications on the client) would be a lot of work, while 2 (send a list of mentions in the clear) could solve most of the problem, and the work to support special treatment of mentions (to avoid trying to infer it from a dodgy display-name match) is something we probably want to do anyway.

[aaronraimist]

They do now

[jryans]

Yes, I think it's safe to close this. If there are issues with the current implementation, let's open separate issues to track that.