Send new DeSo transactions over AMQP to a message broker (aka a firehose)

[mvanhalen]

This PR adds a string to configure in the env file or yml called "AMQP_PUSH_DEST" / "amqp_push_dest" which enables logic to drop new DeSo blockchain TX to a message queue when blocks are committed. It will wait till the node is fully in sync to start pushing and set a flag that AMQP push is enabled. This basically creates a firehose of DeSo transactions

Any AMQP standard supported message broker should work. I used a Rabbit MQ container on the same node to drop the messages. These brokers can be subscribed to and receive a push of new transactions as they are written to the blockchain and the broker queue.

The advantage of this is that you don't need a lot of storage, no TxIndex is needed and no state consumer or handler is needed. The message brokers are super efficient and there is almost no overhead. Our server rad for a few days now without any issues or crashes.

It ignores some transactions types like validator rewards and focuses on user generated transactions

The message data is in easy to read JSON. the broker can be on another server as well. The message values can be bit different since it comes from the source.

• PublicKeys will be compressed and miss the first bytes
• Post Body content is in base64
• Post hashes are in in binary

A queue called block_txns will be automatically created. The queue can be durable or transient this is up to you. The message queue can ack a received tx so you don't get it again.

Each message will contain data which contains default fields like:

• PublicKey (the transactor)
• TxnMeta (varies per transaction type)
• TxnTypeJson (an id identifying the transaction type full list)

And the message properties will contain a MessageId which is the TxnHash (which you can use with DeSo explorer)

If you need some help to convert those let me know.

If you want to use this before core merges (maybe they won't) it, you can use the branch and compile a custom core and upload a custom compiled backend with this core to docker.

A Rabbit MQ example to add to your YML mainnet file:

Add this to your backend ENV settings:
- AMQP_PUSH_DEST=amqp://yyy:xxx@rabbitmq:5672/%2F

Add below your mainnet yml docker file.

    image: rabbitmq:3-management
    ports:
      - "5672:5672"    # AMQP protocol port
      - "15672:15672"  # Management UI port (optional)
    environment:
      # Set default credentials (change these as needed)
      - RABBITMQ_DEFAULT_USER=yyy
      - RABBITMQ_DEFAULT_PASS=xxx
    volumes:
      - rabbitmq_data:/var/lib/rabbitmq
    # Optionally, add a healthcheck if you want to ensure RabbitMQ is ready before the app starts
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "status"]
      interval: 10s
      timeout: 5s
      retries: 5
volumes:
  rabbitmq_data: