Migrate stops without error

[martinpichlo]

Hi,

I've backuped some email accounts via backup command in mirror mode. For testing purpose I tried to restore some bigger accounts (more than 5 GB) via migrate command into a temporal test account. Unfortunately it aborts, although the current folder and a lot of others folders, were not restored. Sometimes with and sometimes without any error message.

---

First try: Stop's without any error message. I noted, that the uid e.g. 10278 is not in sync with the counter e.g. 10271/13587.

~/imap-backup $ ./imap-backup.sh migrate [email protected] [email protected] --reset --verbose
...
D, [2024-06-09T07:46:07.867108 #1] DEBUG -- : [Gesendet] uid: 9230 (9225/13587) - 76045 bytes
....
...
D, [2024-06-09T07:53:15.433479 #1] DEBUG -- : [Gesendet] uid: 10277 (10270/13587) - 4070463 bytes
D, [2024-06-09T07:55:28.537342 #1] DEBUG -- : [Gesendet] uid: 10278 (10271/13587) - 1414207 bytes
~/imap-backup $

---

Second try: With second --verbose

~/imap-backup $ ./imap-backup.sh migrate [email protected] [email protected] --reset --verbose --verbose
...
C:
C:
S: * 9992 EXISTS
S: * 9992 RECENT
S: RUBY10671 OK [APPENDUID 1717705200 31382] Append completed (0.098 + 0.217 + 0.096 secs).
D, [2024-06-08T21:47:49.482746 #1] DEBUG -- : [Gesendet] uid: 10000 (9993/13587) - 6286394 bytes
~/imap-backup $

---

Third try: Also I tried to restart the migration. For that I used the same migrate command, without reset option. But in that case, it's ends with duplicate mails. For example my archive contains 661 emails, and 1322 emails afterwards.

~/imap-backup $ ./imap-backup.sh migrate [email protected] [email protected] --verbose
....
D, [2024-06-08T19:22:03.015870 #1] DEBUG -- : Creating IMAP instance: domain.com, options: {:port=>993, :ssl=>{:min_version=>771}}
D, [2024-06-08T19:22:03.288483 #1] DEBUG -- : [Archiv] 661 to migrate
D, [2024-06-08T19:22:03.288974 #1] DEBUG -- : [Archiv] uid: 1 (1/661) - 40732 bytes
D, [2024-06-08T19:22:03.774696 #1] DEBUG -- : [Archiv] uid: 2 (2/661) - 41652 bytes
D, [2024-06-08T19:22:04.050382 #1] DEBUG -- : [Archiv] uid: 3 (3/661) - 37830 bytes

---

PS: Note: I'm using imap-backup within docker container including a wrapper script. I will do some tests without docker as next step.

Who can help?

[martinpichlo]

I also run into trouble on my MacBook, where I've installed imap-backup via home-brew. After the very huge sent folder with over 13.000 emails, were uploaded, it hangs up in the next folder. Now the program just hangs, so that I need to cancel it after long time without any progress.

....
D, [2024-06-09T19:42:45.692934 #80238] DEBUG -- : [INBOX/Avisen] uid: 56 (56/244) - 44285 bytes
D, [2024-06-09T19:42:45.977402 #80238] DEBUG -- : [INBOX/Avisen] uid: 57 (57/244) - 36568 bytes
D, [2024-06-09T19:42:46.261995 #80238] DEBUG -- : [INBOX/Avisen] uid: 58 (58/244) - 18641 bytes
D, [2024-06-09T19:42:46.553522 #80238] DEBUG -- : [INBOX/Avisen] uid: 59 (59/244) - 31520 bytes
D, [2024-06-09T19:42:46.846000 #80238] DEBUG -- : [INBOX/Avisen] uid: 60 (60/244) - 74774 bytes

[joeyates]

Hi @martinpichlo

Thanks for opening this issue.

I suspect that there is a large email after uid 60 that is taking a long time to upload. Could you check network activity?

Also, please activate network logging with --verbose --verbose to see if there are any clues there.

[martinpichlo]

Hi @joeyates,

I will double check the case with uid 60.

Meawhile: Do you may have ideas regarding the aborts in my initial question? I have used --verbose --verbose there.