feat(send queue): allow setting intentional mentions in media captions edits

[bnjbvr]

Fixes #4302.

Now with tests. Will need ruma/ruma#1991 to fully work, so a bump of Ruma.

[codecov]

Codecov Report

Attention: Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.

Project coverage is 85.07%. Comparing base (1480fad) to head (65ee236).
Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/matrix-sdk-ui/src/timeline/mod.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4464      +/-   ##
==========================================
+ Coverage   85.06%   85.07%   +0.01%     
==========================================
  Files         283      283              
  Lines       31748    31749       +1     
==========================================
+ Hits        27005    27011       +6     
+ Misses       4743     4738       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pixlwave]

I've hooked this up in EXI (using createCaptionEdit(caption:formattedCaption:mentions:)) but I'm not sure it is putting the mentions in the correct place. Given an initial event like this:

Original event

{
  "sender" : "@iosdevsim:matrix.org",
  "content" : {
    "body" : "IMG_0800.jpeg",
    "info" : {
      "thumbnail_file" : {
        "iv" : "UxAgYNYo7iwAAAAAAAAAAA",
        "hashes" : {
          "sha256" : "psUJeuUOBFCdAJGLWZX9vBta\/AshoulAIvvKXM9jGtY"
        },
        "key" : {
          "k" : "LmmDfr4LdLXK6sdSn3EYJt1YIxoGdCevlxfZ5czy3DM",
          "ext" : true,
          "key_ops" : [
            "encrypt",
            "decrypt"
          ],
          "kty" : "oct",
          "alg" : "A256CTR"
        },
        "url" : "mxc:\/\/matrix.org\/sfOAOmkcaTiwQbBluNDLEpSH",
        "v" : "v2"
      },
      "xyz.amorgan.blurhash" : "KlEppVo}jDpMofozRlV@WC",
      "h" : 1536,
      "mimetype" : "image\/jpeg",
      "size" : 827130,
      "w" : 2048,
      "thumbnail_info" : {
        "size" : 140373,
        "w" : 800,
        "mimetype" : "image\/jpeg",
        "h" : 600
      }
    },
    "file" : {
      "iv" : "\/6NOVLfD9hEAAAAAAAAAAA",
      "hashes" : {
        "sha256" : "bQ+qGscrZ6yj2na3L0Gm0rd4ctojmHNViXP+NC8L0u8"
      },
      "key" : {
        "k" : "aqgyLr6H5eOEbbNFWV5mHS1If4rn-zcygcnD6gWmCqM",
        "ext" : true,
        "key_ops" : [
          "encrypt",
          "decrypt"
        ],
        "kty" : "oct",
        "alg" : "A256CTR"
      },
      "url" : "mxc:\/\/matrix.org\/WvDDLsiASNGnIKbbPAiiIwqX",
      "v" : "v2"
    },
    "msgtype" : "m.image"
  },
  "origin_server_ts" : 1736186259021,
  "room_id" : "!gqzOvRiQbuudgUVAJy:matrix.org",
  "event_id" : "$zeK0qAMYljNM-LW8AEsrzdINHkRSAQGsIiFjevP5XM8",
  "type" : "m.room.message",
  "unsigned" : {
    "age" : 219,
    "membership" : "join"
  }
}

When editing it to add the caption Hello @Doug it gives me this:

Edited event

Latest Edit JSON

{
  "sender" : "@iosdevsim:matrix.org",
  "content" : {
    "format" : "org.matrix.custom.html",
    "formatted_body" : "Hello <a href=\"https:\/\/matrix.to\/#\/@douge:matrix.org\">@douge:matrix.org<\/a>",
    "m.relates_to" : {
      "event_id" : "$zeK0qAMYljNM-LW8AEsrzdINHkRSAQGsIiFjevP5XM8",
      "rel_type" : "m.replace"
    },
    "file" : {
      "iv" : "\/6NOVLfD9hEAAAAAAAAAAA",
      "hashes" : {
        "sha256" : "bQ+qGscrZ6yj2na3L0Gm0rd4ctojmHNViXP+NC8L0u8"
      },
      "key" : {
        "k" : "aqgyLr6H5eOEbbNFWV5mHS1If4rn-zcygcnD6gWmCqM",
        "ext" : true,
        "key_ops" : [
          "encrypt",
          "decrypt"
        ],
        "kty" : "oct",
        "alg" : "A256CTR"
      },
      "url" : "mxc:\/\/matrix.org\/WvDDLsiASNGnIKbbPAiiIwqX",
      "v" : "v2"
    },
    "msgtype" : "m.image",
    "m.new_content" : {
      "format" : "org.matrix.custom.html",
      "formatted_body" : "Hello <a href=\"https:\/\/matrix.to\/#\/@douge:matrix.org\">@douge:matrix.org<\/a>",
      "file" : {
        "iv" : "\/6NOVLfD9hEAAAAAAAAAAA",
        "hashes" : {
          "sha256" : "bQ+qGscrZ6yj2na3L0Gm0rd4ctojmHNViXP+NC8L0u8"
        },
        "key" : {
          "k" : "aqgyLr6H5eOEbbNFWV5mHS1If4rn-zcygcnD6gWmCqM",
          "ext" : true,
          "key_ops" : [
            "encrypt",
            "decrypt"
          ],
          "kty" : "oct",
          "alg" : "A256CTR"
        },
        "url" : "mxc:\/\/matrix.org\/WvDDLsiASNGnIKbbPAiiIwqX",
        "v" : "v2"
      },
      "msgtype" : "m.image",
      "m.mentions" : {
        "user_ids" : [
          "@douge:matrix.org"
        ]
      },
      "filename" : "IMG_0800.jpeg",
      "body" : "Hello [@douge:matrix.org](https:\/\/matrix.to\/#\/@douge:matrix.org)",
      "info" : {
        "thumbnail_file" : {
          "iv" : "UxAgYNYo7iwAAAAAAAAAAA",
          "hashes" : {
            "sha256" : "psUJeuUOBFCdAJGLWZX9vBta\/AshoulAIvvKXM9jGtY"
          },
          "key" : {
            "k" : "LmmDfr4LdLXK6sdSn3EYJt1YIxoGdCevlxfZ5czy3DM",
            "ext" : true,
            "key_ops" : [
              "encrypt",
              "decrypt"
            ],
            "kty" : "oct",
            "alg" : "A256CTR"
          },
          "url" : "mxc:\/\/matrix.org\/sfOAOmkcaTiwQbBluNDLEpSH",
          "v" : "v2"
        },
        "xyz.amorgan.blurhash" : "KlEppVo}jDpMofozRlV@WC",
        "h" : 1536,
        "mimetype" : "image\/jpeg",
        "size" : 827130,
        "w" : 2048,
        "thumbnail_info" : {
          "size" : 140373,
          "w" : 800,
          "mimetype" : "image\/jpeg",
          "h" : 600
        }
      }
    },
    "filename" : "IMG_0800.jpeg",
    "body" : "* Hello [@douge:matrix.org](https:\/\/matrix.to\/#\/@douge:matrix.org)",
    "info" : {
      "thumbnail_file" : {
        "iv" : "UxAgYNYo7iwAAAAAAAAAAA",
        "hashes" : {
          "sha256" : "psUJeuUOBFCdAJGLWZX9vBta\/AshoulAIvvKXM9jGtY"
        },
        "key" : {
          "k" : "LmmDfr4LdLXK6sdSn3EYJt1YIxoGdCevlxfZ5czy3DM",
          "ext" : true,
          "key_ops" : [
            "encrypt",
            "decrypt"
          ],
          "kty" : "oct",
          "alg" : "A256CTR"
        },
        "url" : "mxc:\/\/matrix.org\/sfOAOmkcaTiwQbBluNDLEpSH",
        "v" : "v2"
      },
      "xyz.amorgan.blurhash" : "KlEppVo}jDpMofozRlV@WC",
      "h" : 1536,
      "mimetype" : "image\/jpeg",
      "size" : 827130,
      "w" : 2048,
      "thumbnail_info" : {
        "size" : 140373,
        "w" : 800,
        "mimetype" : "image\/jpeg",
        "h" : 600
      }
    }
  },
  "origin_server_ts" : 1736186346868,
  "room_id" : "!gqzOvRiQbuudgUVAJy:matrix.org",
  "event_id" : "$DPRjzziczvrf4nVAu6v7ZrmjTP0WsCvWbAB7i0a5Wfk",
  "type" : "m.room.message",
  "unsigned" : {
    "age" : 621028
  }
}

There was no mention on EX or EW shown in the room list. I would have thought the m.mentions would need to be directly inside of content to work, but it's a while since I looked at edited events. Also sidenote: I had to force quit EX (or maybe just reload the timeline I didn't try that) to get the Latest Edit JSON to show in the View Source screen on EX.

Edit: Ok yeah, just sending a plain text edit (in EW at this stage) with a mention reveals that the m.mentions should be in both the content and new_content:

m.text message edited to add a mention copied from EW

{
  "type": "m.room.message",
  "content": {
    "msgtype": "m.text",
    "body": "* Testing Alice ",
    "m.new_content": {
      "msgtype": "m.text",
      "body": "Testing Alice ",
      "format": "org.matrix.custom.html",
      "formatted_body": "Testing <a href=\"https://matrix.to/#/@iosdevsim:matrix.org\">Alice</a>",
      "m.mentions": {
        "user_ids": [
          "@iosdevsim:matrix.org"
        ]
      }
    },
    "format": "org.matrix.custom.html",
    "formatted_body": "* Testing <a href=\"https://matrix.to/#/@iosdevsim:matrix.org\">Alice</a>",
    "m.mentions": {
      "user_ids": [
        "@iosdevsim:matrix.org"
      ]
    },
    "m.relates_to": {
      "rel_type": "m.replace",
      "event_id": "$vm3DF95wXUFIutgSgVYE9YYHTj2AX5xijtEemUc05uk"
    }
  }
}

[bnjbvr]

Thanks for testing Doug! There's a bug in Ruma, so this is the best the SDK can do at the moment. Will fix on the Ruma side.

[andybalaam]

Looks good, thanks!