Unfortunately, CPU usage with ktorrent 3.3 rc1 and DHT enabled is insanely high.

Code: Select all

17310 modax     20   0  539m  94m  15m S 70.6  9.4  85:56.43 ktorrent

If I disable DHT in Configure KTorrent -> Bittorrent, it goes down to 5%. If I reenable DHT and/or restart KTorrent it, it stays low for some time. So to sum up, not too easy to reproduce.

I don't remember anything like this with 3.3 beta 1.

Do you use IP filter? I do...
maybe is this, however disabling DHT cpu goes to normal like in ktorrent for kde3. Does capture from dht port helps?
I can send mine.

No, I don't have ipfilter enabled.

I did some investigation. Get first callgrind.out from here. I can't claim for sure that I caught this bug (due to huge callgrind overhead I can't measure real ktorrent CPU usage) but there is really something wrong about DHT stuff topping profile list and such numbers like:

Code: Select all

58%        21 966 QList<dht:KBucketEntry>::contains()
48% 58 270 987 dht:KBucketEntry::operator==()
29% 58 770 718 KNetwork::KSocketAddress::operator==()


Yes, operator==() gets called over 58 milion times in around 5 minutes. That's a sign of inefficient algorithm in use at very least (or kbucket QList grows way too large).

After around 13 more minutes, situation is:

Code: Select all

68%             55 841 QList<dht:KBucketEntry>::contains()
56.5% 169 076 507 dht:KBucketEntry::operator==()
33.5% 170 184 705 KNetwork::KSocketAddress::operator==()


So "called" growth seems to be linearly proportional to time.

That list is limited in size to 20 entries (before the changes it used to be 8 ), this just seems to be the result of the number of packets coming in. That list is searched for every packet which comes in.

Will have to investigate this further though, there are probably some optimizations we can do in those comparisons.

George wrote:That list is limited in size to 20 entries (before the changes it used to be 8 ), this just seems to be the result of the number of packets coming in. That list is searched for every packet which comes in.

Will have to investigate this further though, there are probably some optimizations we can do in those comparisons.

Are you sure? QList<dht:KBucketEntry>::contains() (22k) calls ht:KBucketEntry::operator==() 58000k times. That is roughly 2600 operator==() calls per contains() call. Given contains() is O(N) (which I bet it is), that is ~2600 elements per list in the worst case.

Looking more closely, it seems to me that the problem is not in the routing table (which is fine, the buckets don't get to big), but probably in the todo and visited lists of KBucketEntry during a NodeLookup task.

The changes removed several limits in the NodeLookup tasks which make them go on longer, resulting in bigger lists. I'm gonna change the lists into sets so that the contains calls will be O(log N)

Try latest trunk or stable branch and see if you still get the high CPU usage, it should be gone now.

For now it's working ok with dht.

I just registered here and wanted to add that i experience high cpu usage, and now i see that this bug already seems to be fixed

Thank you very much George, i will try the latest svn

it is still a problem here with 3.3rc1. CPU usage is about 15-20% on dual core cpu.

This bug was fixed in 3.3

yep, I've updated ktorrent and see that now it uses 8-10% CPU on the same system. Still very huge value, but it is acceptable now.

Thanks for your good work.

Is it common for 3.31 to take twice as much cpu as 3.25? With 3.31 it never dips below 12% for me and it is always spiking up to 24% all the time. 3.25 stays pretty much between 4-8%. I am not running any plugins at all except the infowidget plugin. Using Kubuntu 9.10 and KDE 4.3.4.

u can try to use some cmd code to check where the packets are coming from.

yvangkwheng wrote:u can try to use some cmd code to check where the packets are coming from.

No idea what that has to do with anything. Ktorrent 3.3.2 still twice the cpu that 3.2.5 is. 12-20 percent.with 3.3.2, 5-8 percent with 3.2.5.