Reply to topic

Dolphin very slow when opening folders with a lot of files

User avatar MirceaKitsune
Registered Member
Posts
327
Karma
0
OS
Dolphin is very slow and laggy when it comes to loading folders that have a lot of files in them. This is most palpable in gallery folders where I keep more than 40.000 images. Whenever I access such a folder, a progress bar shows and it takes about 15 seconds for it to finish. Until then Dolphin is extremely slow and barely usable. This happens both when exploring folders with Dolphin as well as saving / loading files with it's file browser by other applications (eg: right clicking an image in FireFox and saving it, which opens up the little file viewer). Enabling or disabling Previews in such folders does not make any difference.

Is there a setting or solution to make Dolphin load all folders faster? In Windows I don't have such issues with Windows Explorer. If not, will something to improve large folders be done? Can I do anything to fix it in the meantime?
User avatar google01103
Manager
Posts
6668
Karma
25
you can try up'ing to 4.9.1 there was a major re-working of Dolphin, openSuse has a set of 4.9 repos available

Is this folder formated as ntfs?


OpenSuse Leap 42.1 x64, Plasma 5.x

gedgon
Registered Member
Posts
55
Karma
0
OS
google01103 wrote:you can try up'ing to 4.9.1 there was a major re-working of Dolphin, openSuse has a set of 4.9 repos available

Is this folder formated as ntfs?


I can reproduce this behavior in Dolphin 2.1 (form KDE 4.9.1) on ext4. Additionally, https://bugs.kde.org/show_bug.cgi?id=306290
User avatar MirceaKitsune
Registered Member
Posts
327
Karma
0
OS
Yeah, it's a folder on a NTFS partition. Anyway, if I can safely, I might try updating Dolphin and see if that improves anything. Is there something else I could try... like maybe a setting to cache file lists or something so it won't take so long?
gedgon
Registered Member
Posts
55
Karma
0
OS
MirceaKitsune wrote:like maybe a setting to cache file lists or something so it won't take so long?


It won't help. It seems like dolphin spending all the time assigning proper icon to file mime type. Here's how it looks with already cached folder (there is no I/O activity): http://youtu.be/_Rk5JdpKjLo .
User avatar MirceaKitsune
Registered Member
Posts
327
Karma
0
OS
gedgon wrote:
MirceaKitsune wrote:like maybe a setting to cache file lists or something so it won't take so long?


It won't help. It seems like dolphin spending all the time assigning proper icon to file mime type. Here's how it looks with already cached folder (there is no I/O activity): http://youtu.be/_Rk5JdpKjLo .


Nice... it handles 40.000 images quite nicely there! Here it would have taken a lot more I think. Does the "already cached folder" feature exist in KDE 4.8.5, or will it happen with 4.9 and the new Dolphin?
gedgon
Registered Member
Posts
55
Karma
0
OS
MirceaKitsune wrote:Nice... it handles 40.000 images quite nicely there! Here it would have taken a lot more I think. Does the "already cached folder" feature exist in KDE 4.8.5, or will it happen with 4.9 and the new Dolphin?


Cached folder means visited before. Bear in mind, it's quite fast CPU (2500K), and still one core is eaten alive for a couple of minutes.
User avatar MirceaKitsune
Registered Member
Posts
327
Karma
0
OS
gedgon wrote:
MirceaKitsune wrote:Nice... it handles 40.000 images quite nicely there! Here it would have taken a lot more I think. Does the "already cached folder" feature exist in KDE 4.8.5, or will it happen with 4.9 and the new Dolphin?


Cached folder means visited before. Bear in mind, it's quite fast CPU (2500K), and still one core is eaten alive for a couple of minutes.


So Dolphin already caches folders it visited? Since even if I re-visit a folder after some time, it still takes a while to open. Maybe because I add images to that folder, so it needs to re-cache from the beginning each time. Any settings I can tweak for the caching to see if improves anything?
User avatar MirceaKitsune
Registered Member
Posts
327
Karma
0
OS
I recently switched most of my partitions from NTFS to ext4. This helps the problem a lot and large folders open much faster now... but it's still very slow. Will see if there's any change here when KDE 4.10 is out.
User avatar google01103
Manager
Posts
6668
Karma
25
try turning off natural sorting

http://freininghaus.wordpress.com/2012/ ... lphin-2-2/
Loading a folder with many files can be very slow. It turns out that much of this slowness is due to the “natural sorting” of the items, which makes sure that “image10.jpg” comes after “image9.jpg”, and not between “image1.jpg” and “image2.jpg”. There have been some ideas how the “natural comparison” of two file names can be made faster, but this requires a lot of work. In KDE 4.10, we still use the old comparison function, but make use of all CPU cores on your system to keep the delay when opening a large directory short. Look at the final version of the patch to see how little code you need to make Mergesort use multiple threads with QtConcurrent!


OpenSuse Leap 42.1 x64, Plasma 5.x

User avatar MirceaKitsune
Registered Member
Posts
327
Karma
0
OS
google01103 wrote:try turning off natural sorting

http://freininghaus.wordpress.com/2012/ ... lphin-2-2/


Where do you do that in KDE 4.9.5 and Dolphin 2.1?
User avatar google01103
Manager
Posts
6668
Karma
25
settings -> configure Dolphin -> general -> natural sorting of items


OpenSuse Leap 42.1 x64, Plasma 5.x

User avatar MirceaKitsune
Registered Member
Posts
327
Karma
0
OS
google01103 wrote:settings -> configure Dolphin -> general -> natural sorting of items


Thanks. I turned that off, and it seems to be a little faster now. Before it would take about 5 seconds, now it takes nearly 4. So one second is all it improved, but that's good too.
freininghaus
Moderator
Posts
224
Karma
1
OS
The problem might be that we do more re-layouting than necessary when a preview for a file is generated. We should actually only update the icon for this particular file in that case, but when I last did some profiling on large folders, I had the impression that all other items in the directory get updated as well, which makes the CPU quite busy as long as previews keep coming in.

I'll try to have a look at this issue, but I cannot make any promises when this will be - there's always too many bugs and too little time to work on them.
freininghaus
Moderator
Posts
224
Karma
1
OS
I investigated this a bit and found a problem in the code that looks easy to solve, so I hope that we can include this in KDE 4.10.1:

https://git.reviewboard.kde.org/r/108984/

After the folder has been loaded, either previews are being generated or, if previews are disabled, the MIME type of all files is determined asynchrously. At the moment, every time we get a new preview or a MIME type (which determines the icon shown for the file), we indeed trigger a full re-layouting of all items.

There are some more inefficiencies in that area, but this patch is at least a first step towards making Dolphin behave better :)

 
Reply to topic

Bookmarks



Who is online

Registered users: Bing [Bot], eight-bit, Google [Bot], Lachu, mixtou, sirjitsu, Yahoo [Bot]