Hello,

I got severe issues with my mail setup, which became impractical after a long history of heavy usage of mailing lists.
I guess the problem starts on migration of kmail to kmail2 or the switch to akonadi/mysql, I don't know exactly, but I think with the upgrade from lucid to precise.
I hoped, that mail filtering performance might increase with the upgrade to xenial, or at least with the switch to the kubuntu backports ppa, but unfortunately that wasn't the case.

I tried several times this wiki to get around my issues, because akonadictl fsck && akonadictl vacuum didn't solve my problems.

Clean start after a failed migration
https://docs.kde.org/stable5/en/pim/kma ... ation.html
https://docs.kde.org/stable5/de/pim/kma ... ation.html (de)

I also copied my Maildirs to a SSD and symlinked them to $HOME/Mail and ~/Maildir. Also I managed to delete the dupes from a spool mbox file with 2.2GiB size and >230000 emails to retain ~130000 unique mails in a mbox file of 1.4GiB, which I converted to Maildir by mb2md and symlinked it to the same SSD. Also I symlinked the akonadi folders to the SSD and deleted every single akonadi config and cache folders.

Now it seems to work sort of, but all processes seem to be single threaded(?), right? So mysqld, kontact, akonadi_mailfilter and akonadi_maildir are using 100.0 % CPU, but only one core per program. Are there plans to implement multiprocessing?

I just noticed an email in kdepim ML from Martin Steigerwald, that akonadi will implement a new multithreading method(?).

Fix ItemRetriever in case of concurrent requests for the same item(s) | David Faure <faure@kde.org> 2017-02-15 07:43:09 (GMT)
https://cgit.kde.org/akonadi.git/commit ... 3cf0d09bee

Summary:
- ItemRetrievalManager must emit requestFinished() for those other requests,
otherwise the list of pending requests in ItemRetriever is never emptied

- ItemRetriever must not assume that a signal being emitted by
ItemRetrievalManager is necessarily about the request it's waiting for, it
could be for another one. So it must first check in its list of pending
requests to determine whether it should react or not.

With multithreaded unittest, checked for races with clang+tsan.
(There is one race, the connect to ItemRetrievalRequest vs the emit
in other threads, we should lock mLock before connect...)

Maybe such implementation for filters would be great too?

Also, I think on upgrade from 8GiB RAM to 24GiB RAM, Would that help with my performance issue? Maybe I also need a much more performant cpu

Filtering 34 of 3557 Messages takes a long time, but I have to manage filtering of ~170000 emails, 130000 of the mailbox converted maildir (pop3 with fetchmail), and ~44000 in my IMAP account at web.de.

Code: Select all

20:05 CET, Filtering message 188 of 3557
20:15 CET, Filtering message 216 of 3557

These are 28 messages per 10 minutes, that equates to ~170/h, makes 4000/d. So It would take around 42 days for 170000 messages.
That would no big issue, if I could mark all messages in one single step and let my computer process all messages in once, but I think, that would crash the mailfilter process. I got currently 8GiB RAM and 8GiB SSD swap space and a 4 core CPU (not that fast). I have to mark every hour ~4000 messages to filter. That would not be practical for me at all.

German: https://forum.ubuntuusers.de/topic/kmai ... leme-bei-/

Code: Select all

Ubuntu 16.04.2 LTS
kontact 5.2.3 4:16.04.3-0ubuntu2~ubuntu16.04~ppa63
Akonadi 5.2.2  4:16.04.3-0ubuntu2~ubuntu16.04~ppa63
mysql-server 5.7.17-0ubuntu0.16.04.1

Gigabyte Q77M-D2H, Intel(R) Core(TM) i5-3475S CPU @ 2.90GHz, 8GiB RAM

Code: Select all

$ swapon -s
Filename                                Type            Size    Used    Priority
/dev/sda5                               partition       8388604 5828136 -1
$ sudo hdparm -i /dev/sda5
/dev/sda5:
 Model=SanDisk SDSSDX240GG25, FwRev=R201, SerialNo=123323400991

Kind regards, thank you in advance.

That version of the software in Ubuntu is 2 major versions behind. If possible you should test this behavior with a more recent version and see if the issue still persists.

einar wrote:That version of the software in Ubuntu is 2 major versions behind. If possible you should test this behavior with a more recent version and see if the issue still persists.

Hello Einar, I appreciate your answer, thank you! Do you know of a more recent PPA for ubuntu xenial, that I can use to figure out, if a current version of kontact/kmail/akonadi will solve the filter performance issue?

PPA Naming scheme
https://community.kde.org/Kubuntu/Packaging

Staging: (before Testing) [DO NOT USE ON REGULAR SYSTEMS]

Kubuntu Staging KDE Applications
PPA description - Work in progress KDE Applications packages, DO NOT USE ON PRODUCTION SYSTEMS.
https://launchpad.net/~kubuntu-ppa/+arc ... plications

How can I tell which akonadi is provided by the PPA?

Package Version Uploaded by
akonadi 4:16.12.2-0ubuntu1~ubuntu17.04~ppa1 Rik Mills (2017-02-21)

The one you mentioned last is the almost-most recent version out (16.12.3 was released last week). You may want to ask the Kubuntu people about how safe is such a PPA.