MailHog works ineffectively with maildir and large volume of messages

[stanislav-zaprudskiy]

The problem with MailHog on high volumes of messages data (e.g. 16GB) configured with maildir, is that when someone tries to check the e-mail in the UI, which results into the API call on a client’s behalf:

https://example.com/api/v2/messages?limit=1

(UI does limit=50, but even with limit=1 the issue is imminent)

MailHog seems to try to read all the e-mail messages available to it. The corresponding log message is:

Listing messages in /home/mailhog/mount/maildir

After which it echoes every message it reads from maildir folder into logs. At the same time the client request hangs on waiting, and MailHog process continuously grows in memory consumption. Eventually, either client request is terminated by ELB/proxy (when running behind it) or the process gets into OOM and gets restarted/killed - both of which prevent the client from receiving the response from API.

Giving more memory wouldn’t solve the issue, as it seems to be requiring the amount of memory to be equal to amount of e-mail messages data on disk in order to be able to read everything.

Potentially, configuring MailHog against MongoDB might result into different (more effective) behavior, but that produces additional dependency on MongoDB and maintenance overhead associated with its support.

[stanislav-zaprudskiy]

Just found these, which could be related:

• Do not limit messages when using maildir storage driver #178
• fix: paginate messages for maildir storage storage#8
• Maildir storage missing limit fixed storage#11