APN payload successfully delivered but device doesn't refresh?

[titanism]

Hi there,

Per discussion at nodemailer/wildduck#711 (comment) it seems that the APN payload gets successfully delivered, but the iOS device doesn't refresh mail. The new messages aren't fetched (and it waits until the 15m interval) even though Push is enabled and properly setup with XAPPLEPUSHSERVICE support.

Is there any insight you can share as to why this doesn't work?

Thank you,

[freswa]

Have you checked the on-device settings?

[titanism]

Hi @freswa – yes, push notifications are enabled. But nothing happens. Is there any way to debug on device or something? The payload gets successfully delivered, and I've confirmed the device is at 100% battery with alerts/notifications/badges enabled.

[titanism]

@freswa would be happy to sponsor your efforts in debugging this, not sure if dovecot-xaps-daemon still works with user's custom self-hosted servers even and iOS devices?

[freswa]

So if Apple returns that the device received the push notification, the local apnsd has actually received the message. From that point on there can either be a broken payload, a local rate limit or a battery management limitation that causes the service to not work.

1. Broken payload: Have you patched anything (Dovecot, xapsd, xaps-plugin)? What commits do you use?
2. Local rate limit: The local apnsd doesn't willy-nilly forward all notifications. It protects the users from too much notifications by limiting them for rarely used apps.
3. Battery management limits: The local apnsd may not forward the notification if the device is on low battery. Also it can happen, that the remote push notification cannot be processed by Mail.app when the app itself has no energy budget to be called in the background. This is required, since the notification itself does not contain any information about the received message, but only the information about the inbox and the folder the message has been received on. So the actual UI message is generated by the local Mail.app itself.

My best bet is 1. To avoid 2 and 3 I'd recommend to use a device that is used daily and is not too old.

[titanism]

@freswa to be clear – have you seen this approach with dovecot-xaps-daemon and all work with iOS Mobile Mail App? You mentioned Mail.app – which is desktop I believe, not iOS? The topic for mobile mail is com.apple.mobilemail, not com.apple.mail.XServer.xxxxxxxxxxxxxxx as with this dovecot-xaps-daemon.

[freswa]

I've got this daemon running with the corresponding plugin and it sends me push notifications right now.

You mentioned Mail.app – which is desktop I believe [...]?

Nope. Don't mix up topics and App Names.

[titanism]

@freswa Sorry, just to be really clear, your iOS device gets the push notification and refreshes the mailbox? So this is possible on iOS even without a license like Fastmail/Yahoo has?

It must just be something with my payload or rate limiting or device 🤦

[freswa]

Sorry, just to be really clear, your iOS device gets the push notification and refreshes the mailbox?

Yes.

[titanism]

@freswa Thank you so much, I'm going to try again here shortly and deploy all of our work to production and try again.

Also another note, the hostname in the CSR doesn't matter right? I have imap.forwardemail.net right now as the hostname, but should it instead be forwardemail.net (?) – it doesn't have to match the domain name on the IMAP account in iOS mail config, right?

[titanism]

@freswa Something interesting I thought I'd share RE: subfolders other than inboxes – is that I found this repository where the user (an Apple employee) had an extra param in their payload for the subfolder.

https://github.com/argon/push_notify/blob/05b3d8025b217694e45eab8202f3d460f9237652/lib/controller.js#L48

{aps: {"account-id": accountId, m: [mailboxHash]}}

where mailboxHash is the md5 hash of the mailbox name, e.g. INBOX

Not sure if that's the trick to getting subfolders to work?

[titanism]

@freswa Does the APN request payload body need to be a Buffer/Byte or can it be Content-Type: 'application/json' and a stringified body?

I noticed that no matter what I put in in request body (e.g. an empty payload, or no account-id) it always returns successful delivery.

[titanism]

@freswa Does the APN request payload body need to be a Buffer/Byte or can it be Content-Type: 'application/json' and a stringified body?

I noticed that no matter what I put in in request body (e.g. an empty payload, or no account-id) it always returns successful delivery.

Actually, we might just simply be missing some headers per https://github.com/sideshow/apns2/blob/54928d6193dfe300b6b88dad72b7e2ae138d4f0a/client.go#L216-L236, ignore that comment for now!

[freswa]

Was about to point you to that client.go.
Afaik you can send arbitrary payloads. They don't get checked. From Apple's pov you're authenticated with your App-specific certificate. So any payload will only end up in your app code and nowhere else.

Only for standard notifications, there is a specified body format you have to adhere to, since the app is not woken up, but the notification goes straight to your notification center.

[titanism]

Seeing this parse-community/node-apn#114 which seems to be the culprit.

[freswa]

@titanism Just curious. Any particular reason why you don't use this project to send the actual notifications? It scales to hundreds of thousand of simultaneous users....

[titanism]

I tried a minimal curl example, and here was the result, but still, no notification on the device:

❯ ./test.sh
* Host api.push.apple.com:443 was resolved.
* IPv6: (none)
* IPv4: 17.188.170.28, 17.188.171.160, 17.188.171.219, 17.188.170.30, 17.188.169.223, 17.188.170.159, 17.188.170.31, 17.188.171.220
*   Trying 17.188.170.28:443...
* Connected to api.push.apple.com (17.188.170.28) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, CERT verify (15):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: C=US; ST=California; O=Apple Inc.; CN=api.push.apple.com
*  start date: Jun 19 16:31:31 2024 GMT
*  expire date: Apr 10 00:00:00 2025 GMT
*  subjectAltName: host "api.push.apple.com" matched cert's "api.push.apple.com"
*  issuer: CN=Apple Public Server RSA CA 12 - G1; O=Apple Inc.; ST=California; C=US
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha1WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://api.push.apple.com/3/device/REDACTED_DEVICE
* [HTTP/2] [1] [:method: POST]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: api.push.apple.com]
* [HTTP/2] [1] [:path: /3/device/REDACTED_DEVICE
* [HTTP/2] [1] [user-agent: curl/8.9.0]
* [HTTP/2] [1] [accept: */*]
* [HTTP/2] [1] [content-type: application/json; charset=utf-8]
* [HTTP/2] [1] [apns-expiration: 1722527444]
* [HTTP/2] [1] [apns-topic: com.apple.mail.XServer.8fe24cc3-0b15-42b7-8463-82eaeaca6631]
* [HTTP/2] [1] [content-length: 100]
* [HTTP/2] [1] [content-type: application/x-www-form-urlencoded]
> POST /3/device/REDACTED_DEVICE HTTP/2
> Host: api.push.apple.com
> User-Agent: curl/8.9.0
> Accept: */*
> Content-Type: application/json; charset=utf-8
> apns-expiration: 1722612145
> apns-topic: com.apple.mail.XServer.REDACTED
> Content-Length: 100
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 100 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 200
< apns-id: D4017B3D-BC99-9227-F662-D6A9439EBDCF
<
* Connection #0 to host api.push.apple.com left intact

Example script:

curl -v -d '{"aps":{"account-id":"REDACTED_ACCOUNT_ID","m":"7e33429f656f1e6e9d79b29c3f82c57e"}}' \
-H "Content-Type: application/json; charset=utf-8" \
-H "apns-expiration: 1722612145" \
-H "apns-topic: com.apple.mail.XServer.REDACTED" \
--http2 \
--cert /path/to/certificate.crt \
--key /path/to/key.pem \
https://api.push.apple.com/3/device/REDACTED_DEVICE

7e33429f656f1e6e9d79b29c3f82c57e is md5 sum of INBOX

[freswa]

Omit the m parameter for now. Also apns-push-type: alert is not correct. That's the type of notification which ends up in the notification center instantly. Either use background or omit it, which does the Xserver implementation.

[titanism]

It looks like apns-push-type: alert is set in dovecot-xaps-daemon though by default to 'alert' according to logic in https://github.com/sideshow/apns2/blob/54928d6193dfe300b6b88dad72b7e2ae138d4f0a/client.go#L235

[titanism]

Even with m removed, it still does not trigger a notification or badge update. I'm at 100% battery on latest iOS with iPhone 12.

Any other tips to debug or further test @freswa?

[freswa]

> Content-Type: application/json; charset=utf-8
> apns-expiration: 1722612145
> apns-topic: com.apple.mail.XServer.REDACTED
> Content-Length: 100
> Content-Type: application/x-www-form-urlencoded

See first and last line. Looks fishy to me.

Do you actually have any new mail in your Inbox? Because without an unseen mail, there will be no notification either.

[titanism]

Sorry I copy and pasted incorrectly (had to fix headers at first) and then tried to update the example above and pasted wrong sections. Here's a clean example:

❯ ./test.sh
* Host api.push.apple.com:443 was resolved.
* IPv6: (none)
* IPv4: 17.188.143.232, 17.188.179.6, 17.188.143.8, 17.188.178.197, 17.188.143.138, 17.188.143.106, 17.188.143.74, 17.188.178.168
*   Trying 17.188.143.232:443...
* Connected to api.push.apple.com (17.188.143.232) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Request CERT (13):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Certificate (11):
* TLSv1.3 (OUT), TLS handshake, CERT verify (15):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / RSASSA-PSS
* ALPN: server accepted h2
* Server certificate:
*  subject: C=US; ST=California; O=Apple Inc.; CN=api.push.apple.com
*  start date: Jun 19 16:31:31 2024 GMT
*  expire date: Apr 10 00:00:00 2025 GMT
*  subjectAltName: host "api.push.apple.com" matched cert's "api.push.apple.com"
*  issuer: CN=Apple Public Server RSA CA 12 - G1; O=Apple Inc.; ST=California; C=US
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha1WithRSAEncryption
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://api.push.apple.com/3/device/REDACTEd
* [HTTP/2] [1] [:method: POST]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: api.push.apple.com]
* [HTTP/2] [1] [:path: /3/device/REDACTEd]
* [HTTP/2] [1] [accept: */*]
* [HTTP/2] [1] [content-type: application/json; charset=utf-8]
* [HTTP/2] [1] [apns-expiration: 1722527444]
* [HTTP/2] [1] [apns-topic: com.apple.mail.XServer.REDACTEd]
* [HTTP/2] [1] [content-length: 61]
> POST /3/device/REDACTEd HTTP/2
> Host: api.push.apple.com
> Accept: */*
> Content-Type: application/json; charset=utf-8
> apns-expiration: 1722527444
> apns-topic: com.apple.mail.XServer.REDACTEd
> Content-Length: 61
>
* upload completely sent off: 61 bytes
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/2 200
< apns-id: 1F55FB54-4D95-AF56-6798-A8DAA7E54F8F
<
* Connection #0 to host api.push.apple.com left intact

And yes been testing with actual new mail in inbox (I have Thunderbird also open at same time to confirm it already has arrived before I trigger the script).

[titanism]

Does it only trigger if the iOS Mail app is opened (even if running in background/minimized)?

[freswa]

It should trigger when the App is closed, but that depends on what I wrote here.

[freswa]

@titanism Have you double checked the aps-topic in the IMAP response to the XAPPLEPUSHSERVICE cmd?
And is Inbox subscribed in the iOS settings? It's also a good idea to trigger re-register while debugging. Simply add or remove a folder from the push subscription.

[freswa]

If that doesn't work, please try to use the Go implementation to rule out any phone issues.

[titanism]

@titanism Have you double checked the aps-topic in the IMAP response to the XAPPLEPUSHSERVICE cmd? And is Inbox subscribed in the iOS settings? It's also a good idea to trigger re-register while debugging. Simply add or remove a folder from the push subscription.

@freswa the aps-topic in IMAP response to the XAPPLEPUSHSERVICE is com.apple.mobilemail:

Our response to an XAPPLEPUSHSERVICE command:

* XAPPLEPUSHSERVICE aps-version "2" aps-topic "com.apple.mobilemail"

We also respond with an OK of course after writing that payload.

Ref:

• https://opensource.apple.com/source/dovecot/dovecot-293/dovecot/src/imap/cmd-x-apple-push-service.c.auto.html
• https://github.com/st3fan/dovecot-xaps-plugin/blob/3d1c71e0c78cc35ca6ead21f49a8e0e35e948a7c/xaps-imap-plugin.c#L158-L166

Screenshots of my device:

[titanism]

@freswa Do we need to respond with aps-topic value set to the one like com.apple.mail.XServer.xxxxxxxxxxxxxxx?

[freswa]

@freswa Do we need to respond with aps-topic value set to the one like com.apple.mail.XServer.xxxxxxxxxxxxxxx?

Yes, you do. Otherwise someone could trigger notifications with a different certificate on your device.

[titanism]

@freswa Makes complete sense, hours spent when one line of code needed fixed 😆

Will report back once we test again! (working on fix now)

[titanism]

It works!!!! Thank you so much. We have folder support too via m param (md5 sum!) 🚀

See nodemailer/wildduck#711 (comment)