Hello again!

My baloo settings have not been transfered to plasma 5.2 for reasons beyond me.

In systemsettings I find a tab for excluding folders which is empty and lost all settings I had added there before in 4.13.
Now I could simply recreate my settings, including/excluding folders, refining custom file patterns but there are no options anymore whatsoever...

Why, after all the hot and very heated discussion of last summer, about baloo not being transparent and forcing users to use supposedly optimal settings which on the other hand lack individuality, - why, after all that debate, all settings have been removed without any hint, no further documentation, not even a "coming soon" is, well, disturbing... I hope very much I won't have to start over again, editing the baloorc, recreating all GBs of indices...

The result for me at the moment is that my baloo database is about 8,5 GB huge and as the settings (also the visual representation of the settings) disappeared I have no idea whether I have to recreate the index now or is there a way to clean it up of unwanted stuff or is this a bug anyway and it should never be so huge?

It's about 6,5 Gb for emails which is bigger than my local email folders (less than 3GB).

Please can anyone give me some perspective here:
- will there be detailed settings again?
- how big is the database to be expected
- if the file is that big, should it be located on another disk drive with "no-copy-on-write" instead of the BTRFS/SSD it is now, I have no idea how much data is written, modified or created within these 8,5 GB baloo indices at any given time...
- how can I cleanup the index from all unwanted data? (I am not sure whether backups of customer data have been indexed for example...)

Thx for your help,
piedro

p.s.: this is KDE 4.14.12 with plasma 5.2 on ARCH, fully updated

As far as I know concerning the settings, baloo4 reads $HOME/.kde4/share/config/baloofilerc while baloo-frameworks stores its settings in $HOME/.config/baloofilerc.

You could view these two files and edit $HOME/.config/baloofilerc if required.

With baloo-frameworks on ARCH with KDE4 workspace, the kcm module still shows my previous baloo4 settings, but it still edits $HOME/.kde4/share/config/baloofilerc.

For the rest, I don't know.

thx, the new file location is correct and in my case the sparse configuration dialog in systemsettings edits the baloofilerc at the new location in ~/.config/"...

So I guess the old one should be deleted for now...


First part solved,
thx nmset,
p.

piedro wrote:Hello again!

My baloo settings have not been transfered to plasma 5.2 for reasons beyond me.

No settings have been transferred between 4 and 5.

In systemsettings I find a tab for excluding folders which is empty and lost all settings I had added there before in 4.13.
Now I could simply recreate my settings, including/excluding folders, refining custom file patterns but there are no options anymore whatsoever...

You can just duplicate the old config file?

Why, after all the hot and very heated discussion of last summer, about baloo not being transparent and forcing users to use supposedly optimal settings which on the other hand lack individuality, - why, after all that debate, all settings have been removed without any hint, no further documentation, not even a "coming soon" is, well, disturbing... I hope very much I won't have to start over again, editing the baloorc, recreating all GBs of indices...

As with most open source projects, debates are nice, but it ultimately comes down to someone doing the work. If you want something done, come and contribute. Also, coding isn't the only way. You could just work with the VDG and prepare mockups, research how other systems are doing this, see how usable it is for different personas, and different kind of users. There is much to be done.

The result for me at the moment is that my baloo database is about 8,5 GB huge and as the settings (also the visual representation of the settings) disappeared I have no idea whether I have to recreate the index now or is there a way to clean it up of unwanted stuff or is this a bug anyway and it should never be so huge?

It's about 6,5 Gb for emails which is bigger than my local email folders (less than 3GB).

* The email part is still from akonadi_baloo_indexer (a horrible name, since we aren't actually sharing much). I cannot comment much about the size of it w.r.t your maildir size. It should be smaller, but many things could be going on.

* There was never any visual representation of the settings beyond what you are seeing. I'm not sure what you're complaining about.

* If you want to clear up the file part. You can run `balooctl disable`. It will delete the entire file index. You can enable it again after that. `balooctl enable`

* 2 Gb for your files doesn't sound that large. It totally depends on how many files you have.

Please can anyone give me some perspective here:
- will there be detailed settings again?

Someone needs to implement it. Also, Baloo never had detailed settings.

- how big is the database to be expected

Depends on the number of files you have. For 5.3, we are working on a better engine which will reduce the size greatly. That will also require reindexing

- if the file is that big, should it be located on another disk drive with "no-copy-on-write" instead of the BTRFS/SSD it is now, I have no idea how much data is written, modified or created within these 8,5 GB baloo indices at any given time...

How does that make a difference?

- how can I cleanup the index from all unwanted data? (I am not sure whether backups of customer data have been indexed for example...)

For the file part you can run `balooctl disable`.

Hello vHanda!


Thx for your help, for baloo and your clarifications.

No settings have been transferred between 4 and 5.

I think the config file should be transferred somwhow otherwise a user upgrades KDE and then baloo is running for a while with wrong settings (for example starts indexing a formerly excluded backup folder of projects or customers files...) ...

You can just duplicate the old config file?

Thx for your hint. I have done that and put it in the new location (as someone also pointed out).
I suggest a notification during the upgrade process informing users about the changing location (or just copying the old file to the new...)
Anyway, thx, should work now...

As with most open source projects, debates are nice, but it ultimately comes down to someone doing the work. If you want something done, come and contribute. Also,
coding isn't the only way. You could just work with the VDG and prepare mockups, research how other systems are doing this, see how usable it is for different personas, and different kind of users. There is much to be done.

In the systemsettings of KDE 4 there has been the option to exclude folders, include folders (very useful for shared user folders outside the $HOME path for example) and another KCM module for including/exculding file filters (maybe those were done by someone else).

The discussion I was referrering to was a valid complaint of community members about giving users no choice to switch baloo off or tweak it to their needs.
Your reply was (and rightfully so!) to hint to the configuration file and you or someone else added the option to turn file indexing of via UI (which was basically just adding the users home folder to the list of exclusions).

That has been a good solution after a healthy debate (which also is a contribution imho). Now these options vanished again from the systemsettings. Rendering the preceding debate a waste of time. That is the point of my critique and I am not saying you removed it. I am just saying after reaching a solution via debate, having a new version ignoring this solution again is not the right way...

There was never any visual representation of the settings beyond what you are seeing. I'm not sure what you're complaining about.

Also, Baloo never had detailed settings.

There was a second search module in the systemsettings that offered settings for filtering files, including folders and a tickbox to turn email indexing off.
At least on ARCH this had been included in KDE without any additional installation by the user (as far as I remember...)...

If you want to clear up the file part. You can run `balooctl disable`. It will delete the entire file index. You can enable it again after that. `balooctl enable`

Thanks. I will do that immediately.

if the file is that big, should it be located on another disk drive with "no-copy-on-write" instead of the BTRFS/SSD it is now, I have no idea how much data is written, modified or created within these 8,5 GB baloo indices at any given time..

How does that make a difference?

Btrfs' CopyOnWrite has problems with huge constantly changing files like virtual machine images or big database files. The general recommendation is to put these on another disk or turn of COW. Otherwise the system will slow down significantly due to heavy disk activity...
Well, I'll turn COW off for the baloo file just to be on the safe side.

Is there a way to clean up that email indexing part?
Could I just delete the baloo files in ~/.local/share/baloo/email to let them be rebuild?

BTW: the email index searching with kmail works now really well - it's really fast and accurate - so thx for that also!


Phew, long post, sry,
cheers, piedro

piedro wrote:

No settings have been transferred between 4 and 5.

I think the config file should be transferred somwhow otherwise a user upgrades KDE and then baloo is running for a while with wrong settings (for example starts indexing a formerly excluded backup folder of projects or customers files...) ...

Perhaps. A lot of settings needs to be migrated.

As with most open source projects, debates are nice, but it ultimately comes down to someone doing the work. If you want something done, come and contribute. Also,
coding isn't the only way. You could just work with the VDG and prepare mockups, research how other systems are doing this, see how usable it is for different personas, and different kind of users. There is much to be done.

In the systemsettings of KDE 4 there has been the option to exclude folders, include folders (very useful for shared user folders outside the $HOME path for example) and another KCM module for including/exculding file filters (maybe those were done by someone else).

Right. That was my code before I changed it to the new design. One person took the old code and released the advanced KCM.

I'm a little burned out with the whole KCM discussion. We really discussed the simpler one a lot, and there was very mixed feedback and some really rude people. After that I haven't felt like touching it, cause I would just get screamed at.

Even now, I don't know ..

The discussion I was referrering to was a valid complaint of community members about giving users no choice to switch baloo off or tweak it to their needs.
Your reply was (and rightfully so!) to hint to the configuration file and you or someone else added the option to turn file indexing of via UI (which was basically just adding the users home folder to the list of exclusions).

In the first version of Baloo (1.5 years ago) , there wasn't a clear disable button. But then it was added within a month and we do have the command line tools.

That has been a good solution after a healthy debate (which also is a contribution imho). Now these options vanished again from the systemsettings. Rendering the preceding debate a waste of time. That is the point of my critique and I am not saying you removed it. I am just saying after reaching a solution via debate, having a new version ignoring this solution again is not the right way...

The advanced KCM isn't part of the Baloo releases. It probably hasn't been ported to KF5.

Also, Baloo never had detailed settings.

There was a second search module in the systemsettings that offered settings for filtering files, including folders and a tickbox to turn email indexing off.
At least on ARCH this had been included in KDE without any additional installation by the user (as far as I remember...)...

The email settings were when we had Nepomuk. Once that was killed, the new KCM was always very sparse.

Btrfs' CopyOnWrite has problems with huge constantly changing files like virtual machine images or big database files. The general recommendation is to put these on another disk or turn of COW. Otherwise the system will slow down significantly due to heavy disk activity...
Well, I'll turn COW off for the baloo file just to be on the safe side.

I'll have to look more into it. I don't know much about this.

Is there a way to clean up that email indexing part?
Could I just delete the baloo files in ~/.local/share/baloo/email to let them be rebuild?

BTW: the email index searching with kmail works now really well - it's really fast and accurate - so thx for that also!

Yes, and delete .kde4/share/config/baloorc.

Email indexing is in a very strange place right now. It was developed as part of Baloo because we could re-use a lot of the code, but it is for all intents are purposes completely independent. We should have just named it `akonadi_indexer` instead of adding baloo over there.

Anyway, now that we're no longer releasing any Baloo-Qt4 releases, nothing is going to change - No additional disable options or bug fixes. Also, since we're not going to have an Akonadi Qt5. (Akonadi is being rewritten). The Baloo code will be eventually discarded.

Thank you very much for your thorough answer!

I get a bit scared hearing that akonadi will be rewritten (I still remember the nightmares when it was introduced) but probably that is a good thing.

The Btrfs issue is only relevant if there are huge files with constant changes. If the file indexer takes (in my case) around 2GB that is no problem at all, at least as far as I know.
The email indexer with these 6,5GB could be... but please, do not bother, as you mentioned it will be replaced...

For the KCM part, I still think the baloo system would benefit from a simple KCM with
- include/exclude folders (I think the "include" would be such a versatile addition!)
- an UI way to add/modify/delete file masks for filtering
- a clickbox to trigger balooctl start/stop (plus indexing for emails or whatever else it's used)

For me as a user it's just nice to actually SEE the new stuff somewhere (who ever reads a manual?) and it feels very good to know "If it scares me I could turn it off"...
Wouldn't yell at you though...

Not much more to say then
- I will delete the old config files,
- baloo is running smoothly,
- emails are a mess anyway no matter what baloo does with them or doesn't...

So thx again, keep on dancing the baloo shuffle,

cheers, piedro