How the persistence work on EMQX?

[badsector3]

Hi.

I have one core node of EMQX with persistent EFS volume attached to the node (AWS). I use Kafka bridge integration.

  coreTemplate:
    spec: 
      replicas: 1
      volumeClaimTemplates:
        storageClassName: emqx-storage-class
        volumeMode: Filesystem
        resources:
          requests:
            storage: 20Gi
        accessModes:
          - ReadWriteMany

For the test, I removed access from EMQX -> Kafka, so I could see 25 massages in the queue.
Then I restarted the pod and re-establish the connection back from EMQX to Kafka.
I would expect the messages to be sent and appear in the Kafka topic, but that was not the outcome.

Should the messages be dropped on the EFS while in queue? What do I miss here?

[badsector3]

Found information that for data persistency I should use RockDB.
Highly Reliable MQTT Data Persistence Based on RocksDB
Does the operator support the persistency with RockDB?

[Rory-Z]

Sorry, this feature isn't release, please keep follow us.

[badsector3]

Thanks. Any dates when that will be released?

[Rory-Z]

For the test, I removed access from EMQX -> Kafka, so I could see 25 massages in the queue.
Then I restarted the pod and re-establish the connection back from EMQX to Kafka.
I would expect the messages to be sent and appear in the Kafka topic, but that was not the outcome.

Could you please check the Buffer Mode of the config ? you can check the documents: https://docs.emqx.com/en/enterprise/latest/data-integration/data-bridge-kafka.html#advanced-configurations

[badsector3]

Thanks @Rory-Z That works fine. Will test the latency, but I assume it will be quite a bit 🤔