oprofile results (1 week)
Page 1 of 1 (10 posts)
Tags:
None
Registered Member 
|
Notes:
ktorrent-svn, 2048 kbps down, 1600 kbps up (mostly up used), KDE 3.5.6 (svn, haven't updated in a couple of months but should be irrelevant). qt-copy from 3.5.6 branch.
ktorrent easily takes a minimum 15% on my box the entire time it's running - and I don't think it should be. It should be mostly IO bound with occasional cpu time spent on hash verification (which really shouldn't take much cpu in the grand scheme of things). I've tried sliding the CPU bar from slow to fast to middle, etc. Doesn't seem to make a difference. Also have tried GUI updates of 5 seconds and that also seems to make little difference. As you can see from the top graph - the majority of time is spent within QT itself. |
|
Registered Member
|
Simple top output:
Footprint seems quite large for what the app should typically be caring about. Even if one were to take the 50 seeding tasks I have going, apply a generalized 64k chunk size, w/ an average of 1GB per torrent, utilize a 32 byte table entry struct for each chunk w/ 16 bytes of malloc overhead: 1024*1024*1024/65536 * 50 * (32+16) 39321600 Would be 40MB for a generalized table of structs tracking chunk checksum and chunk location. Now I don't know if ktorrent is even doing this presently (or why it would have to in the first place - once a torrent has been leeched it shouldn't change - but I'm presuming that rechecking is happening per the option in preferences). That leaves another 60MB of footprint for other things (in my particular case) if I were seeding 50GB of torrents w/ 64k chunksize (which I'm not - but it's a nice worse case). Just restarted entirely:
I know not to care much about virt - particularly because as far as I can tell kt is using mmap (presumably to map files for seeding/leeching a given chunk area): $ nm -A /usr/local/kde/bin/ktorrent /usr/local/kde/lib/libktorrent-2.2dev.so | egrep 'm(un)*map' /usr/local/kde/lib/libktorrent-2.2dev.so: U mmap64@@GLIBC_2.1 /usr/local/kde/lib/libktorrent-2.2dev.so: U munmap@@GLIBC_2.0 BTW: I notice that there are two renditions of mmap class wrappers in the util and torrent dirs, one for standard purposes and one for cachefiles. Also, the cachefile variant seems to be using mmap for some purposes and read/write syscalls for others. I'm sure some optimization can be done there. Also ::growFile should be using fruncate() and not a raw loop of 1k writes(). fruncate() already allocates and extends (with zero padding) or shrinks a file to a provided size in _one_ call. |
|
Registered Member
|
|
|
Registered Member
|
|
|
Moderator
|
I was planning on ditching the chunk based approach and only use pieces (16 KB each), but that is something for the KDE 4 port.
The problem with chunks is that you keep an entire chunk in memory, while most of it can already be placed on disk. For example during a chunk download, if you downloaded the first 10 pieces, you can calculate the hash up to that point and save those 10 to disk and remove them from memory. But these are rather big changes, and we are not gonna do these anymore. The plan is to release the 2.2 version in the middle of june and then fully focus on the KDE 4 port. Btw, ftruncate fails on FAT. And growFile should not be used much. (disk preallocation takes care of that) Also assuming 64 KB as the average chunk size, is in my opinion a rather big underestimation. |
|
Moderator
|
The CPU usage for libktorrent seems to be pretty normal, considering the fact that you are mostly uploading.
The Qt one is a bit weird, I mean ManualPeerSource::qt_invoke, that is just part of a dialog to add peers, assuming you didn't constantly had this open, I would not expect that many samples. |
|
Registered Member
|
Well I'd contest that normal is defined by how things presently are - but I am of the opinion that there is definitely room for optimization. Also the op results are a bit hazy - after doing some additional profiling with cachegrind - which btw shows a big fat SHA1Generate block (further pointing back to caching of this data as a win). I didn't run the cachegrind for a significant period of time (just about 10 minutes) but I'll be restarting it later. About the only thing that's different with my setup vs normal is that mine is using NX as the X server rather than a straight X server. But the dialogs haven't been open - just the main screen and minimized 90% of the time as well. That's what I also found odd. |
|
Moderator
|
There is certainly room for optimization. But the results where as I would expect in a mostly upload situation.
We know, SHA1 hash generation can take up quite a bit of CPU time, certainly when the chunks are big. That is why we added the incremental calculation mode for bigger chunks. Big chunks resulted in a drop in download speed and a CPU usage spike when a chunk was ready.
Something I need to look in, but first we are gonna release a 2.2 beta (probably tomorrow) |
|
Registered Member
|
You're gonna hate me - but I've worked out some of the ktorrent load.
I had been using my Intel PIIX IDE/ATA drivers as a module - which specifically limited using DMA on my drives - i.e. *none*. After ironing that out, it went from 5M/sec to 55M/sec HD throughput. While ktorrent still hovers around in general, it's not nearly as invasive - and I'm sure you could see why having my HD access method in the trashcan would surely hurt things. 
|
|
Moderator
|
Page 1 of 1 (10 posts)
Bookmarks
Who is online
Registered users: Bing [Bot], Google [Bot]
