Hi!

Sadly this problem (which has been there since Kmail for KDE 4) is still around:

When I check the maintainance tab within folder properties of any of my mail folders - I get incomplete indices for the email indexing. (e.g. 216 objects in folder, only 12 included in the index).

Turning off the indexing for this folder and later turning in on again doesn't remedy any of this - it stubbornly stays at 12 indexed objects.

This renders the whole searching for mail and search folder functionality in kmail completely useless.

For the old version of kmail there has been the idea to delete the folder ~/.local/share/baloo/email and restart baloo via balooctl. Well this folder doesn't exist anymore and I doubt that email index is stored within the regular index file created by baloo at all. Btw: the baloo file index is seems to be complete.

Another suggestion has been to open akonadi console and deltet the cahces for all the email folders.
With the hint that this could lead to data loss if the files in the cache have never been written to the mail folders in the first place. Clearly not the way to go...

1. So, please, anyone, where does the akonadi indexing agent store it's files nowadays (restarting this agent via akonadi console shows no difference...)?

2. How to re-initiate the indexing process non-destructively to get all my mails into the database. I tested local mysql database and also the sqlite version, I'd like to stick with the local mysql storage since the mysql server is running anyway...

thx for reading,
p.

You can restart indexing by editing ~/.config/baloorc and setting initialIndexingDone to false. Restart either the indexing agent (from akonadiconsole) or Akonadi as a whole to start the process.

Hi, thx for your help!

In baloorc I have this line:

[Akonadi]
agentIndexingVersion=4

Is this correct? There shouldn't be any akonadi 4 anymore, should it?

Last time I asked in the arch forums, I've been told that KF5 and Plasma5 doesn't contain any KDE4 components anymore...

Could you clarify this?

I am confused...

BTW: Where do I find the documentation on the configuration options?

ty, p.

That version has nothing to do with the library (KF5/kdelibs version), it's provided so that in case the indexing format changes (for whatever reason) it can be adjusted to force a reindexing.
There is (IIRC) no configuration documentation on indexing, because it's not exposed to the user interface (so far).

Thank you for the insight!

Well, my problem though does not disappear:
I followed your advice and let the akonadi indexer run for two days straight after setting the entry to "false".

the sad result is that there is not one (!) single folder that is is completely indexed as designated by the maintenance tab...

Also right clicking on a mail folder to bring up the context menu doesn't work anymore for many folders in kmail (this is after the latest update. Version is kmail 5.1.3, KF 5.20, QT 5.6 on Arch fully updated... )

So my question remains: How can I reindex my mail folders (local folders and Imap folders)?

thx, p.

Sorry, I've made a mistake - when I re-checked this morning I missed my local folder:

Re-indexing worked on all local folders but on none of my IMAP folders - which clearly is not a problem of the servers since other email programs work fine here (thunderbird for example has no problem with full text search and indexing).

So at least some progress! Though any incomplete index breaks search completely - there is no point in searching if you can't rely on the results being complete.

ty, p.

Now I tried restarting my system...

Now though the indexing clearly is incomplete the indexer does not seem to work at all - it only takes up 6k of RAM (it's been above 200k before, also some CPU activity...) and the CPU doesn't seem to be taxed...

Seriously, I cannot be the only one having problems here.... what is going on?
How can I check the flawless working of the akonadi indexer?
Where to look for any failure logs?
Does it only work on local SQlite server?
Could there by a non standard MySQL server setting that needs to be set?

I tried on the KDE bug tracker but there is so much wrong with akonadi and kmail - it is impossible to find the right bug to report to. Also most of it is probably entirely outdated or based on different versions...

How is the forum here connected with the bug tracker - is there any communication?

I just search for mails with content containing "facebook" in the one folder where all facebook messages come in (216 messages in folder) with the result of one (!) mail found. Checked the index - 2 mails of 216 indexed.

This is really not good and I understand that I cannot expect things to work out of the box but searching and indexing is a critical component - at least I could expect to find some comprehensible information on how to troubleshoot.

desperate, p.

Are you using stored (disconnected/offline) IMAP? Indexing will *not* work if you don't use disconnected mode.

Hmm, actually I never saw the setting for "disconnected Imap".

Is it the checkbox in the account settings saying "download all messages for offline usage"?
This is not checked by default. And I there there has been a number of issues with this setting including including mail loss, empty mail bodies and syncing problems over the last three years.

Is this save to use now? (and is this the setting you are referring to?)

Thx for your help, p.

Yes, that is the one, and I've been using it for quite a while without issues. You won't be able to index mail without it (because if not PIM can't access the full body of the message). You need only to wait if your mail provider uses Courier IMAP as IMAP server: there are two bugs related to Courier's IMAP handling that are problematic. These have been already fixed, but not present in a stable release yet (they will once Applications 16.04 is out).

Hello einar!

Thank you for this information. Let's hope for 16.04... funny how I find myself hoping for something in the next version every month for some reason... that's how my personal KDE experience feels for me the last three years now...

But to the issue:
I have done tests before posting and the results are puzzling:
- the agent works at the designated time
- the mail is stored in the "drafts" folder of the account until it's send
- the mail is kept in the "sent" folder of the account once it's sent

So my guess would be that the agent doesn't get along with my box when I am absent.

Like interfering with some energy saving profile (though I disabled any energy saving as far as the UI allows),
something crashing in Kontact (quite unstable atm, e.g. crashing on closing kmail a lot)
or some problem with the mysql server being connected for a long time (there has been a problem with akondi reconnecting to the mysql server after server time out limits...)

Since there never is any documentation on any of this (for example there is the option to use local sqlite, external mysql or postgres server connection for akonadi but I cannot find any explanation about benefits or problems with any one of those, how to confirm it's working and whether additional options on the server side are recommended...) for me it's about impossible to troubleshoot or even correctly report on issues like that.

Regarding bug reporting, - I stopped doing that if the bug is not very obvious and very specific.
Bug reporting is quite time consuming, the inbuilt bug reporting tool in KDE never worked correctly for me, and the results are simply frustrating. I've been reporting for over 5 years with countless bugs:

Around 80% of them have never got any devs comment (though users vote support), issues fixed are often not reported back to the system - resulting in all these report corpses that seem to be still open issues.

More than half the bugs never seem to be even looked at, never get confirmed and to be honest, the whole environment is not very user friendly or respectful (with some notable exceptions for which I am very grateful - in these cases the whole bug solving process can actually be satisfying and fun!)

But the biggest issue with bug reporting is the structure of the system which, again, lacks good explanation.
Searching for problem with imap sending emails in kmail produces hundreds of hits, most of them completely unrelated but the bug reporting user should find a possible duplicate?

Also the report has to be assigned to a certain project and the correct component of this software, seriously, how should any user know?

That's just turning people away (I know some)... at least it does for me.

So thanks a lot for your continued help. If we can figure out what's wrong with the send later agent I will gladly write a bug report. If we can't figure where the actual problem sits - any bug report on this is very likely to be just another waste of time and a dead entry in the bug database...

Now I will wait for 16.04 apps and report back if the problem still remains.

Sorry for the long post.
Ty, piedro

Hello once more!

One follow up question:

I changed all accounts to the disconnected offline mode now.
What does the account setting "switch to offline mode when ending" mean exactly?
(that's probably a bad translation of the setting but I cannot switch kmail to english anymore without changing the whole KDE language setting, I might miss something here - this worked before in the "help" menu as "switch application language"...)

thanks, p.

Now I have set all accounts to download messages for offline use.

After that I stopped akonadi with "akonadictl stop".

Then I've edited ~/.config/ballorc as mentioned above recognice the initail indexing hasn't been done.

Then I waited a whole night leaving things run.

Still I have my email account folders only show I tiny fraction of mails indexed (17 of 1921 mail for example). These numbers do not seem to change at all.

So my question remains: How can I force akonadi indexer to create a fresh full index of my mail folders?

ty, p.

You may also want to wipe ~/.local/share/baloo/email while you adjust the configuration file.

Not directly related, but make sure you have all libkf5akonadi* packages installed.

I just had a other issue with search related things and the problem was that `libkf5akonadisearchxapian5` was not installed.