Inbox only showing new mails but not whole message list

[Chrisi-Bavaria]

Steps to reproduce

1. Login, open mail app
2. Click on inbox

Expected behavior

Inbox should show all mails

Actual behavior

Inbox is empty or showing only the mails after a particular point in time

Mail app version

3.7.8

Mailserver or service

Cyrus IMAPd

Operating system

Debian

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database

MariaDB

Additional info

Mailbox was showing all mails earlier. I then deleted a mail from inbox using a parallel logged in Thunderbird client. Now mail app shows only mails coming in after this incident. Currently 28 mails are in this inbox, mail app is showing an empty inbox. No errors in log. OCC mail-account-sync is showing no errors:

$ sudo -u www-cloud php8.2 --define apc.enable_cli=1 occ mail:account:sync 5 -f -vvv
[debug] Skipping mailbox sync for 68
[debug] Skipping mailbox sync for 77
[debug] Skipping mailbox sync for 62
[debug] Skipping mailbox sync for 75
[debug] Skipping mailbox sync for 67
[debug] Skipping mailbox sync for 63
[debug] Skipping mailbox sync for 66
[debug] Skipping mailbox sync for 70
[debug] Skipping mailbox sync for 69
[debug] Skipping mailbox sync for 71
[debug] Skipping mailbox sync for 64
[debug] Skipping mailbox sync for 73
[debug] Syncing 61
[debug] Locking mailbox 61 for new messages sync
[debug] Locking mailbox 61 for changed messages sync
[debug] Locking mailbox 61 for vanished messages sync
[debug] Running initial sync for 61
[debug] Initial sync 5:INBOX - mailbox meta search took 1s. 30/30MB memory used
[debug] Range for findAll did not find any (not already known) messages and all messages of mailbox INBOX have been fetched.
[debug] Initial sync 5:INBOX - fetch 3 messages from IMAP took 0s. 30/30MB memory used
[debug] Initial sync 5:INBOX took 1s
[debug] Unlocking mailbox 61 from vanished messages sync
[debug] Unlocking mailbox 61 from changed messages sync
[debug] Unlocking mailbox 61 from new messages sync
[debug] Skipping mailbox sync for 65
[debug] Skipping mailbox sync for 80
[debug] Skipping mailbox sync for 74
[debug] Skipping mailbox sync for 78
[debug] Skipping mailbox sync for 72
[debug] Skipping mailbox sync for 79
[debug] Skipping mailbox sync for 76
[debug] Building threads for account 5
[debug] Account 5 has 1293 messages with threading information
[debug] Threading 1293 messages - build ID table took 0s. 31/31MB memory used
[debug] Threading 1293 messages - build root container took 0s. 31/31MB memory used
[debug] Threading 1293 messages - free ID table took 0s. 31/31MB memory used
[debug] Threading 1293 messages - prune containers took 0s. 31/31MB memory used
[debug] Threading 1293 messages - group by subject took 0s. 31/31MB memory used
[debug] Threading 1293 messages took 0s
[debug] Account 5 has 667 threads
[debug] Account 5 has 0 messages with a new thread IDs
31MB of memory used

SELECT * FROM oc_mail_messages WHERE mailbox_id = 61
returns zero results.

[Chrisi-Bavaria]

Okay, if it is somehow difficult to investigate on this... Is there an official way to reset the caching of the mail app, so that the user's inbox will be read from IMAP again? Is there an occ command or any safe sql?

[ChristophWurst]

If you enable debug mode (not advised for production environment) you have a option to clear the cache of a mailbox from the context menu of the Mail app.
Otherwise there is no mechanism for this

[Chrisi-Bavaria]

Okay, I enabled debug-mode when the instance was idling and found the context menu entry:
DELETE https://nextcloud/apps/mail/api/mailboxes/61/sync
...but it looks like the API call is really only working in debug-mode, I get "unauthorized", when calling in non-debug-mode.

But more important: Clearing the cache like described solved the issue: All mails are now visible in the Inbox.

So as long as this sync is still somehow unstable when looking at other sync issues here in the tracker... Would it be possible to always show this menu entry? It will cause some load at huge mailboxes on both servers, but it's better than missing mails, right? Or make it available via OCC...

[ChristophWurst]

So as long as this sync is still somehow unstable when looking at other sync issues here in the tracker... Would it be possible to always show this menu entry? It will cause some load at huge mailboxes on both servers, but it's better than missing mails, right? Or make it available via OCC...

Not necessary. #10038 will address this automatically.

[mat-m]

@ChristophWurst said:

Not necessary. #10038 will address this automatically.

Well, if it's done only once a week, we may need to be able to trigger this more often.

My scenario is different: I currently have unified inbox issues when I delete files in the dedicated inbox through Nextcloud Mail, and go back to Unified. Mail is still in the list but I get "Mail not found" when I click on it.
Clicking on the reload button in Unified do not consolidate the list neither. Are we on the same root cause (and I'll wait for the fix) or should I post a new bug ?

[st3iny]

Well, if it's done only once a week, we may need to be able to trigger this more often.

There are fixes that improve the sync logic, so that you (hopefully) won't even see vanished emails any longer. Furthermore, we will implement a periodic background job that runs weekly to clean already existing "ghost" emails. Lastly, there will also be an option in the frontend to "repair" a mailbox manually in case the weekly job is not good enough or users are impatient.