Just after logging in from KDM, KDE is snappy. It's fast, responsive, and wonderful. Alt-tabbing is instant, scrolling through my files in Dolphin is lighting fast, scrolling through folder view is fast, menus and dialogs pop up without delay ... etc.

However after a good hour of usage it starts to slow down significantly. Even after closing all applications, suspending strigi indexing, and disabling compositing there is a delay when trying opening menus, scrolling through files, etc. A quick logout and login brings it back to normal, but it's unacceptable really. top reports no difference in idle CPU usage but sees usage spike up a lot faster when doing simple things like alt-tabbing or moving a window.

I really don't know how to track down exactly what's causing this. Any ideas?

In the past I found with my installation that a similar issue was related to KNotify4 and the workaround I applied was to change system notifications from "Use KDE sound system" to "use external player" and I set that to "mplayer".

Moult wrote:Just after logging in from KDM, KDE is snappy. It's fast, responsive, and wonderful. Alt-tabbing is instant, scrolling through my files in Dolphin is lighting fast, scrolling through folder view is fast, menus and dialogs pop up without delay ... etc.

However after a good hour of usage it starts to slow down significantly. Even after closing all applications, suspending strigi indexing, and disabling compositing there is a delay when trying opening menus, scrolling through files, etc. A quick logout and login brings it back to normal, but it's unacceptable really. top reports no difference in idle CPU usage but sees usage spike up a lot faster when doing simple things like alt-tabbing or moving a window.

I really don't know how to track down exactly what's causing this. Any ideas?

Might be a hardware problem, I have the same Experience before, I have to Use another Cooling Fan to dissipate the heat.

The problem is probably related to your distro rather than to KDE. It has to do with the "vm.swappiness".

For instance, Ubuntu and Ubuntu based distros usually have vm.swappiness set to a default of "60" which basically means that once 40% of your physical ram is used up the system begins to swap out everything used above and beyond that into a disk based paging file and as you know, hard drive access is dead slow compared to physical ram.

The additional problem with this is that once you close all your open programs so all that you have is a desktop, these all the files/functions/resources that were previously swapped out to the disk are not returned to the system. This is the reason that even just opening menus or windows can be slow due to their function being called from the disk rather than memory. By default, the kernel setting for releasing these disk based caches is more appropriate to a server environment than a home desktop environment* (see reference link below for more info).

So what we have here is two separate problems seen in a desktop environment. System functions being swapped out to a disk based cache too soon and then not released back into the system once windows and programs are closed.

The idea behind this is fine when used in a server environment but not so great when the OS is used as a desktop system.

Fortunately, the fix is rather easy and speeds things right up. It involves modifying the "/etc/sysctl.conf" file.

First, make a backup of the "/etc/sysctl.conf" file and then, as root, open "sysctrl.config" into a text editor and add (or modify if the lines already exist) the following 2 lines to the bottom of the file:

This line sets swapping out to the disk when the system is down to 20% physical memory:

Code: Select all

vm.swappiness=20

And this line adjusts when and what type of cache the kernel releases first (speeds up the browsing of files):

Code: Select all

vm.vfs_cache_pressure=50

Add the two above lines to the bottom of the "sysctrl.config", save and reboot. You're system should be much snappier now.

Hope this helps. If not, please let me know.

*Reference: http://rudd-o.com/en/linux-and-free-sof ... o-fix-that

Note on above link: In the tutorial it recommends setting vm.swappiness to "1" which I wouldn't recommend as it leaves virtually no physical memory left before it swaps additional functions out to the disk (which is why I suggested setting swappiness to 20%).

FWIW when I had the issue with KNotify my vmswappiness was and is set to 0.

esdaniel wrote:FWIW when I had the issue with KNotify my vmswappiness was and is set to 0.

Strange that. For me, KDE 4.3.2 through KDE 4.4 hasn't suffered from any slow down problem. At least none that I've seen that wasn't related to vm.swappiness being set at too high a value. But that's just my experience and hardly reflects other setups.

It could depend on the distro that KDE is running on or like mhuktar stated, it could be related to hardware in some cases (or both?). This is where controlled user testing would come in handy. Something like the folks at Mozilla use to root out the bugs in an upcoming release. That way you'd be conducting testing across a broad range of configurations instead of doing a poke and hope of single user setups.

There I go rambling again. Sorry about that.

OK I've followed a few tips here and there. I've changed the notifications to mplayer and I've reduced swappiness from 60 to 30. However most importantly I've changed my swap from a swapfile to a partition (previously used a swapfile because I didn't exactly plan well when setting this box up and didn't have enough space to hibernate with it).

So far all is good, but this sort of things happens over time so I'll update with how this is going.

Note that Global Shortcut keys and notifications can wake up all applications connected to the D-Bus session bus, which if the system is partially swapped won't help.

OK It's gone back to being slow. I did a few tests. I'm not too good with these system stuff so feel free (no pun intended) to correct me and tell me what insight I'm missing or what I really should be looking for. After a fresh login this is what free -m reports:

Code: Select all

             total       used       free     shared    buffers     cached
Mem:          1753        976        776          0         31        329
-/+ buffers/cache:        615       1138
Swap:         1023          0       1023

and after using it for quite some time, having kmail, akregator, firefox, korganizer, and amarok running it gets rather lagtastic. Here's the new free -m report:

Code: Select all

             total       used       free     shared    buffers     cached
Mem:          1753       1739         14          0         40        655
-/+ buffers/cache:       1042        711
Swap:          494          2        492

Now closing all the applications only manages to free up a little bit, which is a bit disappointing:

Code: Select all

             total       used       free     shared    buffers     cached
Mem:          1753       1665         88          0         39        635
-/+ buffers/cache:        990        763                                 
Swap:          494          5        489

So yes, it's still quite laggy. I take a peek at htop and apparently there aren't a lot of high mem % users, but there are a lot of processes which are identical and all take up 1%, 1%, 1%, etc - and that adds up. Here are some of the main ones:

Code: Select all

/usr/bin/akonadi_ical_resource --identifier akonadi_ical_resource_##
akonadi_maildir_resource
akonadi_vcard_resource
akonadi_contacts_resource
kdeinit4: kio_trash [kdeinit] trash local:/tmp/ksocket-dion/klauncher...

(each of those are repeated about 10-20 times)

Killing all of them as well as a few others and all my apps and services (eg: mysql, apache, quasselcore) really helps, as shown by this free -m:

Code: Select all

dion@epicbox ~ $ free -m
             total       used       free     shared    buffers     cached
Mem:          1753       1217        536          0         43        679
-/+ buffers/cache:        495       1258
Swap:          494          5        489

Even though the numbers don't quite match up to the original value of 776 in terms of responsiveness it seems on par. It's quite easy to lower it back down to a level of 400 though (open up firefox, dolphin) but it's acceptable.

Any ideas why all those processes were stuck there even though there was clearly nothing using them?

It might help for us to get a more detailed snapshot of usage when you've got everything running... when all your apps have been open and the lagginess has taken hold could you run this script and paste the result so we can see the memory usage across the entire process table:

#!/bin/sh

printf "%-6s %-9s %s\n" "PID" "Total" "Command"
printf "%-6s %-9s %s\n" "---" "-----" "-------"

for PID in `ps -e | /usr/bin/awk '$1 ~ /[0-9]+/ { print $1 }'`
do
CMD=`ps -o comm -p $PID | tail -1`
# Avoid "pmap: cannot examine 0: system process"-type errors
# by redirecting STDERR to /dev/null
TOTAL=`pmap $PID 2>/dev/null | tail -1 | \
/usr/bin/awk '{ print $2 }'`
[ -n "$TOTAL" ] && printf "%-6s %-9s %s\n" "$PID" "$TOTAL" "$CMD"
done | sort -n -k2

BTW pmap is an interesting tool for delving deeper into memory usage, you might want to play with that on particular processes.

This little one liner is also good for snapshotting i.e. execute this at a particularly laggy moment if possible:
ps -eo pcpu,%mem,cmd|sort -k2 -r

One thing that I'd also consider doing in this situation is to grab or burn a live CD and run that to compare the difference between a vanilla install and what I'm working with to see if it's an unfortunate case of misconfiguration.

(NB: Proceed with caution) Often a simple get-out-of-jail trick is to use 'mv' to move your ~/.kde to a backup destination and then log back out and back in and have KDE create a fresh profile then you can inspect differences in the configs (~/.kde/share/config/).

With every major KDE upgrade I mv my .kde folder so I'm running as vanilla as possible.

This is the result of your script http://wipup.org/updates/view/36

Here is the snapshot command results, http://wipup.org/updates/view/35 I'm not sure if I got it at a really laggy moment though, I did it when scrolling through dolphin.

Curious to know if you've configured Nepomuk to use Virtuoso backend (http://lists.semanticdesktop.org/piperm ... 00278.html), it looks like it's in use (which is good) from the process list you've shared - and it's busy indexing - if the drive is a slow one such as a laptop drive then that will have a significant performance impact as well.

I'd like the opinion of other more experienced KDEers to chime in on this thread as per info Moult pasted above too, please.

I reran the memory script and made sure strigi wasn't indexing anything. These are the results: http://wipup.org/updates/view/37

I'm also quite sure I'm using virtuoso:

Code: Select all

dion@epicbox ~ $ cat .kde4/share/config/nepomukserverrc
[Service-nepomukmigration1]
autostart=false

[main Settings]
Storage Dir[$e]=$HOME/.kde4/share/apps/nepomuk/repository/main/
Used Soprano Backend=virtuosobackend

I think it's an Akonadi issue but can't confirm based on my limited experience, this link might be worth a look:
http://userbase.kde.org/Akonadi#Common_Problems

Doing akonadictl stop got rid of all the akonadi_resource_* processes I mentioned on the previous page and freed up a wonderful 200MB. killall -9 kio_trash helped significantly too but not as much. I find turning off mysql (needed because I do webdev) helps. Turning off nepomuk entirely (even though of late I have been using the search a bit more frequently) helped a load as well.

Programwise the biggest culprits are Amarok (no surprise - I think I'll have to switch soon to something more lightweight) and krunner (and Firefox too but I can't live without a web browser )