Reload certificates without having to restart the service

[reubenmiller]

Is your feature improvement request related to a problem? Please describe.

thin-edge.io components (such as tedge-agent, tedge-mapper) don't reload certificates automatically when the files are changed on disk. This most likely affects both the local mTLS certificates as well as the cloud device certificate used when using the built-in bridge.

Reloading certificates is typically required when the certificates have been renewed (typically renewed before the certificate expires). Currently the only way to reload the certificate is to restart the associated service, e.g. tedge-agent, tedge-mapper-c8y etc. This is very disruptive as it will shutdown all in-progress actors including the workflow actor.

Describe the solution you'd like

It would be useful to support re-reading the certificates from disk without having to restart the service.

Below are some options for implementation (though we are also open to other ideas/options):

• Option 1: Reload certificates when receiving a SIGHUP signal

• Option 2: Re-read the certificates from disk when invalid certificates are found (or before trying to connect or reconnect to the service which requires the certificates)

Describe alternatives you've considered

Additional context

Below shows the steps which confirms that the certificates are definitely not read from disk after the MQTT has been initialized. Test with the tedge-agent with version 1.3.0:

1. Create certificate for tedge-agent (to connect to mosquitto) to expire in 5 minutes

2. Start the tedge-agent

3. Renew the certificate to expire in 1 hour but don't restart tedge-agent

4. Wait > 5mins for the first certificate to expire

5. Restart mosquitto (to interrupt the connect)

6. Check the logs of the tedge-agent to see if it was able to successfully reconnect to the local MQTT broker

   Oct 02 05:54:09 d4544799dc86 tedge-agent[1313]: 2024-10-02T05:54:09.102719033Z ERROR mqtt_channel::connection: MQTT connection error: I/O: received fatal alert: CertificateExpired
   Oct 02 05:54:10 d4544799dc86 tedge-agent[1313]: 2024-10-02T05:54:10.120175846Z ERROR mqtt_channel::connection: MQTT connection error: I/O: received fatal alert: CertificateExpired
   Oct 02 05:54:11 d4544799dc86 tedge-agent[1313]: 2024-10-02T05:54:11.129288526Z ERROR mqtt_channel::connection: MQTT connection error: I/O: received fatal alert: CertificateExpired
   Oct 02 05:54:12 d4544799dc86 tedge-agent[1313]: 2024-10-02T05:54:12.149244267Z ERROR mqtt_channel::connection: MQTT connection error: I/O: received fatal alert: CertificateExpired
   Oct 02 05:54:13 d4544799dc86 tedge-agent[1313]: 2024-10-02T05:54:13.16724398Z ERROR mqtt_channel::connection: MQTT connection error: I/O: received fatal alert: CertificateExpired
   Oct 02 05:54:14 d4544799dc86 tedge-agent[1313]: 2024-10-02T05:54:14.181756544Z ERROR mqtt_channel::connection: MQTT connection error: I/O: received fatal alert: CertificateExpired

[reubenmiller]

Looks like mosquitto also has planned to support reloading the certificates on SIGHUP, see eclipse-mosquitto/mosquitto#2038