KDE leaking into ~/.xsession-errors is a known bug. Mine was 58 GB at one stage… There is a workaround in the link, although it's been fairly well behaved for me in the last few months (since moving from Ubuntu to Arch).

vHanda wrote:This is a new issue that has cropped up. It seems we are using newer features of Xapian which are causing the database to get corrupted.

The only way to fix it now is to run -
$ balooctl disable
$ balooctl enable

This will reset the database and restart it.

It looks like it's using 100% of a core again. I left it for an hour or so, then got annoyed and ran these commands. It's been going for another hour…

New installation: Baloo takes all RAM after a minute and crashes Plasma. Guess there are enough bug reports around .

I'm not sure if this is related, but when I boot, I get this message, asking for root permission. When I hover over the link, it says "Click to edit org.kde.baloo.filewatch.raiselimit". I've only noticed this in the last week or so, probably since the upgrade from baloo 5.9.2-1 -> 5.13.0-2 on 19 August. This is also a similar time to when I noticed the 100% CPU bug again.

EDIT: I attempted to boot without giving it root permission; I'm not really sure why it should need it. However, the window just kept popping up again.

sparhawk wrote:I'm not sure if this is related, but when I boot, I get this message, asking for root permission. When I hover over the link, it says "Click to edit org.kde.baloo.filewatch.raiselimit". I've only noticed this in the last week or so, probably since the upgrade from baloo 5.9.2-1 -> 5.13.0-2 on 19 August. This is also a similar time to when I noticed the 100% CPU bug again.

EDIT: I attempted to boot without giving it root permission; I'm not really sure why it should need it. However, the window just kept popping up again.

Try this. https://wiki.archlinux.org/index.php/KD ... atch_limit

It's possible this might fix your 100% CPU usage issue as well. It could just be that baloo or some other program isn't able to handle hitting the limit in a graceful fashion.

davidmar wrote:Try this. https://wiki.archlinux.org/index.php/KD ... atch_limit

Thanks for the suggestion. Unfortunately, after a reboot, I still got the popup.

OTOH, the 100% CPU thing seems to have disappeared by itself.

==EDIT==
I added my home directory to the list of directories for baloo to ignore. That stopped the popup on reboot!

It is unacceptable that *TWO* years after this report that the problem persists. There is no excuse. It should not be necessary for me to go googling around to figure out how to fix up the baloo database or disable the stinky thing. If you can't fix it, then GET RID OF THE DAMN THING. Thank you.

i deleted ~/.xSesion whatever is . then fixed problem

Having this same problem.

Running Kubuntu 16.04 LTS 64bit.

After I click to do something I'm normally hit with about 15s of system lag.

baloo_file_extractor is constantly running.

Do I need it? Can I remove it or stop it permanently? I don't need to search for files if indexing is all it does.

TechieAndroid wrote:Having this same problem.

Running Kubuntu 16.04 LTS 64bit.

After I click to do something I'm normally hit with about 15s of system lag.

baloo_file_extractor is constantly running.

Do I need it? Can I remove it or stop it permanently? I don't need to search for files if indexing is all it does.

Can you check the output of the following command, and if it is updated when you download e.g. a new pdf file from the internet?

Code: Select all

qdbus org.kde.baloo /fileindexer org.kde.baloo.fileindexer.currentFile

This returns the file baloo is currently busy on.

Not entirely sure, but I think it is possible to disable baloo_file_extractor specifically by running

Code: Select all

kwriteconfig --file $HOME/.config/baloofilerc --group General --key only\ basic\ indexing true

thanks for the advice! it helped to add

Code: Select all

only basic indexing=true

to General group of $HOME/.config/baloofilerc

ozone wrote:

TechieAndroid wrote:Having this same problem.

Running Kubuntu 16.04 LTS 64bit.

After I click to do something I'm normally hit with about 15s of system lag.

baloo_file_extractor is constantly running.

Do I need it? Can I remove it or stop it permanently? I don't need to search for files if indexing is all it does.

Can you check the output of the following command, and if it is updated when you download e.g. a new pdf file from the internet?

Code: Select all

qdbus org.kde.baloo /fileindexer org.kde.baloo.fileindexer.currentFile

Does it return ~/.xsession-errors? If yes, can you check in ~/.config/baloofilerc if the exclude filter contains already .xsession-errors?

Not entirely sure, but I think it is possible to disable baloo_file_extractor specifically by running

Code: Select all

kwriteconfig --file $HOME/.config/baloofilerc --group General --key only\ basic\ indexing true

This is a tough problem

On Fedora 27 with:

Code: Select all

$ plasmashell --version
  plasmashell 5.12.4
$ kf5-config --version
  Qt: 5.9.4
  KDE Frameworks: 5.44.0
  kf5-config: 1.0
$ rpm --query --file /usr/bin/baloo_file_extractor
  kf5-baloo-file-5.44.0-1.fc27.x86_64


I have two baloo_file_extractor instances running. Each is allocating 100% of a core.

One instance is for the currently logged-in user, one is for another user (not currently logged in).

Both seem to be going nowhere (maybe they are actually mining bitcoin?).

Watching the script

Code: Select all

#!/bin/bash
lsof -p17657 | grep -v "/usr/lib64"


to check which files are held open shows an immutable list, actually identical for both instances:

Code: Select all

COMMAND     PID   USER   FD      TYPE             DEVICE  SIZE/OFF     NODE NAME
baloo_fil 17657 devman  cwd       DIR              253,3      4096 29360129 /home/devman
baloo_fil 17657 devman  rtd       DIR              253,0      4096        2 /
baloo_fil 17657 devman  txt       REG              253,0    133616   940985 /usr/bin/baloo_file_extractor
baloo_fil 17657 devman  mem       REG              253,3    643072 29360216 /home/devman/.local/share/baloo/index
baloo_fil 17657 devman  mem       REG              253,0   8406312  2754395 /var/lib/sss/mc/passwd
baloo_fil 17657 devman  mem       REG              253,0 113049440   918003 /usr/lib/locale/locale-archive
baloo_fil 17657 devman  mem-r     REG              253,3      8192 29360174 /home/devman/.local/share/baloo/index-lock
baloo_fil 17657 devman  mem       REG              253,0      2485  2363837 /usr/share/locale/en_GB/LC_MESSAGES/kcoreaddons5_qt.qm
baloo_fil 17657 devman  mem       REG              253,0       488  2252116 /usr/share/locale/en/LC_MESSAGES/kcoreaddons5_qt.qm
baloo_fil 17657 devman  mem       REG              253,0      3589  2370089 /usr/share/locale/en_GB/LC_MESSAGES/solid5_qt.qm
baloo_fil 17657 devman    0r     FIFO               0,12       0t0  1335098 pipe
baloo_fil 17657 devman    1w     FIFO               0,12       0t0  1335099 pipe
baloo_fil 17657 devman    2w     FIFO               0,12       0t0  1335100 pipe
baloo_fil 17657 devman    3u     unix 0x000000002c523aa1       0t0  1335105 type=STREAM
baloo_fil 17657 devman    4u  a_inode               0,13         0    14592 [eventfd]
baloo_fil 17657 devman    5u  a_inode               0,13         0    14592 [eventfd]
baloo_fil 17657 devman    6u     unix 0x000000005196e127       0t0  1340080 type=STREAM
baloo_fil 17657 devman    7u  netlink                          0t0  1346629 KOBJECT_UEVENT
baloo_fil 17657 devman    8u  a_inode               0,13         0    14592 [eventfd]
baloo_fil 17657 devman    9u     unix 0x000000002337472f       0t0  1342624 type=STREAM
baloo_fil 17657 devman   10r  a_inode               0,13         0    14592 inotify
baloo_fil 17657 devman   11u      REG              253,3    643072 29360216 /home/devman/.local/share/baloo/index
baloo_fil 17657 devman   12r      REG                0,4         0  1340082 /proc/17657/mounts
baloo_fil 17657 devman   13u     unix 0x00000000f5c7101c       0t0  1344237 type=STREAM
baloo_fil 17657 devman   14u     unix 0x000000000277268b       0t0  1344238 type=STREAM
baloo_fil 17657 devman   15r      REG              253,0   8406312  2754395 /var/lib/sss/mc/passwd
baloo_fil 17657 devman   16ur     REG              253,3      8192 29360174 /home/devman/.local/share/baloo/index-lock
baloo_fil 17657 devman   17u      REG              253,3    643072 29360216 /home/devman/.local/share/baloo/index
baloo_fil 17657 devman   18w      REG              253,3    643072 29360216 /home/devman/.local/share/baloo/index

Querying baloo via qdbus yields nothing:

Code: Select all

$ qdbus org.kde.baloo /fileindexer org.kde.baloo.fileindexer.currentFile
Error: org.freedesktop.DBus.Error.NoReply
Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.

Nothing peculiar in the SELinux audit log with

Code: Select all

cat /var/log/audit/audit.log | grep AVC

Just taking a short look with gdb, I'm no longer used to this..

installing debuginfo:

Code: Select all

# dnf debuginfo-install kf5-baloo-file-5.44.0-1.fc27.x86_64

Attach to the process:

Code: Select all

# gdb -p 17657

We get:

Code: Select all

Attaching to process 17657
[New LWP 17658]
[New LWP 17659]
Missing separate debuginfo for /lib64/libKF5BalooEngine.so.5
Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/f5/c279d0a5057aaa5ab58f437b89b79c31dd61e9.debug
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
0x00007f31a4409f9e in Baloo::DocumentUrlDB::get(unsigned long long) const () from /lib64/libKF5BalooEngine.so.5
...
(gdb) bt
#0  0x00007f31a4409f9e in Baloo::DocumentUrlDB::get(unsigned long long) const () from /lib64/libKF5BalooEngine.so.5
#1  0x00007f31a441a834 in Baloo::Transaction::documentUrl(unsigned long long) const () from /lib64/libKF5BalooEngine.so.5
#2  0x0000559628f42878 in Baloo::App::processNextFile (this=0x7ffe2b1880d0) at /usr/src/debug/kf5-baloo-5.44.0-1.fc27.x86_64/src/file/extractor/app.cpp:85
#3  0x00007f31a2586934 in QSingleShotTimer::timerEvent(QTimerEvent*) () from /lib64/libQt5Core.so.5
#4  0x00007f31a257b21b in QObject::event(QEvent*) () from /lib64/libQt5Core.so.5
#5  0x00007f31a31be92c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from /lib64/libQt5Widgets.so.5
#6  0x00007f31a31c6174 in QApplication::notify(QObject*, QEvent*) () from /lib64/libQt5Widgets.so.5
#7  0x00007f31a2551be7 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /lib64/libQt5Core.so.5
#8  0x00007f31a25a1f5b in QTimerInfoList::activateTimers() () from /lib64/libQt5Core.so.5
#9  0x00007f31a25a27f1 in timerSourceDispatch(_GSource*, int (*)(void*), void*) () from /lib64/libQt5Core.so.5
#10 0x00007f319ca1cb77 in g_main_context_dispatch () from /lib64/libglib-2.0.so.0
#11 0x00007f319ca1cf20 in g_main_context_iterate.isra () from /lib64/libglib-2.0.so.0
#12 0x00007f319ca1cfac in g_main_context_iteration () from /lib64/libglib-2.0.so.0
#13 0x00007f31a25a2c2f in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /lib64/libQt5Core.so.5
#14 0x00007f31a255096a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /lib64/libQt5Core.so.5
#15 0x00007f31a2559094 in QCoreApplication::exec() () from /lib64/libQt5Core.so.5
#16 0x0000559628f41a2f in main (argc=<optimized out>, argv=0x7ffe2b1882b8) at /usr/src/debug/kf5-baloo-5.44.0-1.fc27.x86_64/src/file/extractor/main.cpp:57
...
(gdb) info threads
  Id   Target Id         Frame
* 1    Thread 0x7f31a51c1500 (LWP 17657) "baloo_file_extr" 0x00007f31a4409f9e in Baloo::DocumentUrlDB::get(unsigned long long) const () from /lib64/libKF5BalooEngine.so.5
  2    Thread 0x7f31904ee700 (LWP 17658) "QXcbEventReader" 0x00007f31a174cc6b in poll () from /lib64/libc.so.6
  3    Thread 0x7f3186d39700 (LWP 17659) "QDBusConnection" 0x00007f31a174cc6b in poll () from /lib64/libc.so.6

Both instances are in Baloo::DocumentUrlDB::get(). Are they mutually deadlocking? Are they looping in there?
Source from https://api.kde.org/frameworks-api/fram ... ource.html shows a suspicious loop

Code: Select all

  141     while (id) {
  142         auto p = idFilenameDb.get(id);
  143         //FIXME: this prevents sanitzing
  144         // reactivate Q_ASSERT(!p.name.isEmpty());
  145
  146         ret = p.name + '/' + ret;
  147         id = p.parentId;
  148     }


...just a possibility.

Install the debuginfo for libKF5BalooEngine:

Code: Select all

dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/f5/c279d0a5057aaa5ab58f437b89b79c31dd61e9.debug

According to your backtrace, it is currently inside Baloo::DocumentUrlDB::get
On the gdb prompt, do:

1. print the contents of "QByteArray ret":
   (gdb) p/s (char*)(ret.d) + ret.d->offset
2. print the value of the docId:
   (gdb) p/x docId
3. print the id of the current path segment (may not work, as optimized out:
   (gdb) p/x id
4. print the value of its parent
   (gdb) p/x p.parentId

Now, see while it (apparently) loops:

1. set a breakpoint:
   (gdb) br documenturldb.cpp:146
2. do one iteration:
   (gdb) cont
3. check the id of the parent again:
   (gdb) p/x p.parentId

You can repeat 2. and 3. to watch if it shows any progress.

And, when you have gathered the information, please report a bug at https://bugs.kde.org, Product frameworks-baloo, Component engine.

Thanks bruns.

Still wrangling the gdb (baloo again treading water, but the stack trace is different)

You seem to have opened https://phabricator.kde.org/D12335 - "Avoid infinite loops when fetching the URL from DocumentUrlDB".

So my baloo is currently in deep swampland again:

Code: Select all

#0  QConcatenable<QByteArray>::appendTo (out=<error reading variable: access outside bounds of object referenced via synthetic pointer>, ba=<synthetic pointer>...) at /usr/include/qt5/QtCore/qstringbuilder.h:339
#1  QConcatenable<QStringBuilder<QStringBuilder<QByteArray, char>, QByteArray> >::appendTo<char> (out=<error reading variable: access outside bounds of object referenced via synthetic pointer>, p=...)
    at /usr/include/qt5/QtCore/qstringbuilder.h:358
#2  QStringBuilder<QStringBuilder<QByteArray, char>, QByteArray>::convertTo<QByteArray> (this=<optimized out>) at /usr/include/qt5/QtCore/qstringbuilder.h:118
#3  QStringBuilder<QStringBuilder<QByteArray, char>, QByteArray>::operator QByteArray (this=<optimized out>) at /usr/include/qt5/QtCore/qstringbuilder.h:131
#4  Baloo::DocumentUrlDB::get (this=this@entry=0x7ffe101e8a60, docId=docId@entry=68692401262230787) at /usr/src/debug/kf5-baloo-5.44.0-1.fc27.x86_64/src/engine/documenturldb.cpp:145
#5  0x00007fddc6f34834 in Baloo::Transaction::documentUrl (this=<optimized out>, id=id@entry=68692401262230787) at /usr/src/debug/kf5-baloo-5.44.0-1.fc27.x86_64/src/engine/transaction.cpp:104
#6  0x00005594b86b2878 in Baloo::App::processNextFile (this=0x7ffe101e90d0) at /usr/src/debug/kf5-baloo-5.44.0-1.fc27.x86_64/src/file/extractor/app.cpp:85
#7  0x00007fddc50a0934 in QtPrivate::QSlotObjectBase::call (a=0x7ffe101e8b30, r=<optimized out>, this=<optimized out>) at ../../include/QtCore/../../src/corelib/kernel/qobject_impl.h:101
#8  QSingleShotTimer::timerEvent (this=0x5594b90bd8d0) at kernel/qtimer.cpp:318
#9  0x00007fddc509521b in QObject::event (this=0x5594b90bd8d0, e=<optimized out>) at kernel/qobject.cpp:1268
#10 0x00007fddc5cd892c in QApplicationPrivate::notify_helper (this=this@entry=0x5594b902d7f0, receiver=receiver@entry=0x5594b90bd8d0, e=e@entry=0x7ffe101e8dd0) at kernel/qapplication.cpp:3722
#11 0x00007fddc5ce0174 in QApplication::notify (this=0x7ffe101e90a0, receiver=0x5594b90bd8d0, e=0x7ffe101e8dd0) at kernel/qapplication.cpp:3481
#12 0x00007fddc506bbe7 in QCoreApplication::notifyInternal2 (receiver=0x5594b90bd8d0, event=event@entry=0x7ffe101e8dd0) at kernel/qcoreapplication.cpp:1016
#13 0x00007fddc50bbf5b in QCoreApplication::sendEvent (event=0x7ffe101e8dd0, receiver=<optimized out>) at ../../include/QtCore/../../src/corelib/kernel/qcoreapplication.h:233
#14 QTimerInfoList::activateTimers (this=0x5594b90827b0) at kernel/qtimerinfo_unix.cpp:643
#15 0x00007fddc50bc7f1 in timerSourceDispatch (source=<optimized out>) at kernel/qeventdispatcher_glib.cpp:182
#16 0x00007fddbf536b77 in g_main_dispatch (context=0x7fddac004ff0) at gmain.c:3142
#17 g_main_context_dispatch (context=context@entry=0x7fddac004ff0) at gmain.c:3795
#18 0x00007fddbf536f20 in g_main_context_iterate (context=context@entry=0x7fddac004ff0, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at gmain.c:3868
#19 0x00007fddbf536fac in g_main_context_iteration (context=0x7fddac004ff0, may_block=may_block@entry=1) at gmain.c:3929
#20 0x00007fddc50bcc2f in QEventDispatcherGlib::processEvents (this=0x5594b9067ee0, flags=...) at kernel/qeventdispatcher_glib.cpp:423
#21 0x00007fddc506a96a in QEventLoop::exec (this=this@entry=0x7ffe101e9020, flags=..., flags@entry=...) at kernel/qeventloop.cpp:212
#22 0x00007fddc5073094 in QCoreApplication::exec () at kernel/qcoreapplication.cpp:1289
#23 0x00005594b86b1a2f in main (argc=<optimized out>, argv=0x7ffe101e92b8) at /usr/src/debug/kf5-baloo-5.44.0-1.fc27.x86_64/src/file/extractor/main.cpp:57

and seems to be forever looping in

Code: Select all

340     in /usr/include/qt5/QtCore/qstringbuilder.h
339     in /usr/include/qt5/QtCore/qstringbuilder.h

I have not really found an original source repo for Qt, thus code from apt-browse-org

https://www.apt-browse.org/browse/ubuntu/trusty/main/i386/qtbase5-dev/5.2.1+dfsg-1ubuntu14/file/usr/include/qt5/QtCore/qstringbuilder.h

At 340 we find an inlined

Code: Select all

QAbstractConcatenable::convertFromAscii(ba.constData(), ba.size(), out);

I don't know what's going on in there, the first sensible stack frame moving up is

Code: Select all

#4  Baloo::DocumentUrlDB::get (this=this@entry=0x7ffe101e8a60, docId=docId@entry=68692401262230787) at /usr/src/debug/kf5-baloo-5.44.0-1.fc27.x86_64/src/engine/documenturldb.cpp:145
145             ret = p.name + '/' + ret;


The above line can be found in https://api.kde.org/frameworks-api/frameworks-apidocs/frameworks/baloo/html/documenturldb_8cpp_source.html (however at line 154, how do I into obtaining the correct source file, is it time to embed URLs to github into debug information yet?)

So, here we seem to try to concatenate three things using operator '+'....

"ret" is a "QByteArray" (http://doc.qt.io/qt-5/qbytearray.html)
'/' is a char
"p" is marked 'auto' and thus apparently a "IdFilenameDB::FilePath" where the "FilePath" is a struct wrapping a "QByteArray" according to https://api.kde.org/frameworks-api/backup/frameworks/baloo/html/idfilenamedb_8h_source.html. So "p.name" is a "QByteArray".

And trying to concatenate that char and those two "QByteArray" seems to result in non-termination. Weird.

You are apparently using baloo from KF5.44, which does not contain the fix.

Its the responsibility of distributions to either provide current versions of the packages, or at least cherry-pick bugfixes from upstream.