It would be nice to implement caching support in ktorrent. Compared to Vuze with caching enabled and downloading the same torrent @ ~1.2 MB/s, once in Vuze and once in ktorrent, the latter does *a lot* more disk reads and writes than Vuze. It basically trashes the disk every 0.4 or so seconds while Vuze is much better at that. I'm using Ext3 but I guess it will have similar behavior (though not as severe) on extent-based file systems.

Thoughts?

Yeah, cache options would be great. I've got 3 Gigs of RAM, and I wouldn't mind dedicating most of it towards ktorrent. Also, it would be nice if there were a hot-key or GUI button that would allow you to quickly switch between cache profiles, where one profile could allocate most free RAM to ktorrent, and another profile could reign the cache back to a minimal level so that other memory intensive tasks could be run. Then, when the user is finished with those tasks finished, they can quickly and conveniently switch back to the other profile.

In Vuze, I think you have to change the default config to allow the caching of torrents that you are seeding, and this would be useful in ktorrent as well.

Also, Vuze has an option that allows every downloading piece to be written to disk as it is completed, avoiding having to wait for a write until the cache is full. I think that this would have to be a critical feature if there were an also an option to use a large cache size.

Anything that makes use of unused RAM *and* reduces disk reads is definitely worth implementing!

That'd be nice

It basically trashes the disk every 0.4 or so seconds while Vuze is much better at that.

Is that really so bad? i think getting stuff to disk ASAP is a good idea (maybe its just because my laptop sucks), if its getting in the way of other apps then perhaps there should be a config in ktorrent the sets a lower ionice

Wouldn't it be better to pass stuff on down to lower APIs (kernel/kde?) that choose the catching, that way it can respond more dynamically (e.g if i start running firefox with 300+ tabs it gives some memory back), sort of like running with a lower ionice but for writing to disk.

OFC if this can't be done this with current APIs (or is just too much work for the hassle) then this would be useful, however leaving it off by default to stop ktorrent being called a memory hog

Actually, it is bad when you at the same time download a high-speed torrent in the background and stream from the disk 1080p content. I've often noticed hiccups and only when downloading high-speed torrent that trash the disk so fast - it's not related to my Net connection and does not occur when using Vuze for the job. Granted, I can use ionice but it'll be nice to have something in ktorrent that caches in RAM and at set intervals/percentages commits it to disk, just like Vuze does it.

On a related note, I've also noticed that when downloading big torrents (4-10 GB), my swap is used much often than when compared to Vuze. For example, the day before yesterday I've downloaded a 5GB torrent with Vuze (pre-allocation enabled, caching enabled). The only active program during the downloading was Vuze and it never hit the swap throughout the whole process. I repeated this process yesterday but this time using ktorrent (pre-allocation enabled, another torrent but same size & comparable DL speed). When I checked swap usage in the end, it had eaten 160 MB. This is not just a single isolated case and I've seen this behavior in the past too with ktorrent when getting big torrents, it tends to use swap much more often than Vuze with big torrents... I've got 2 GB RAM and 2 GB swap so it's not really that much of an issue, but I'm just wondering how come ktorrent tends to use swap much often than Vuze.

I dunno, caching and then writing it all at once not affecting your playback seams to be getting lucky, what happens if vuze decides to write its whole cache just as your player decides to read the next section of the film? Sure the collision doesn't occur much but given that there are tools to prevent this form happening at all ( setting mediaplayers to a high io priority (probably not realtime but close) and torrent apps to low io priority) wouldn't it be better for ktorrent to use them instead of implementing its own cache.

I think its really distro/tool makers fault for not providing an easy way (that i know of) for me to always run mplayer/xine/etc at a high ionice and ktorrent/vuze/etc at a low ionice (hell have a predefined set of defaults that do it for me), but as a workaround ktorrent could have a setting to increase its own ionice (changing its nice would be a nice feature too)

Anyway I'm not opposed to your idea, i just think that there are better ways of achieve the goal of not having ktorrent interfere with other programs due to its high disk usage, however as i'm not going to be the one coding it its not really my choice, i just hope (at least) one of the solutions is implemented.

yeah, you're right that committing memory from RAM to disk at set intervals will cause problems too but like Shadow Tek mentioned, Vuze has specific options to to deal with such things efficiently and thus doesn't trash the disk like ktorrent. It has three major options which minimize disk trashing

- read-ahead to reduce disk reads when uploading
- cache download data to reduce disk writes and also disk reads required for piece checking
- write complete pieces to disk immediately. it smooths disk action but can result in more write operations

one can also set the amount of disk cache (mine is 20MB) and also other things like maximum write queues and read queues. As I said before, Vuze handles such things much better than current ktorrent and I've done quite an extensive behavior-check comparing the two how they deal with big and high-speed torrents.

George,

Any info if this is on your todo list and will be implemented?

Caching is on the list, will probably happen after I add support for bittorrent over UDP, which is the next big thing I'm going to do after 3.3rc1 is out.

Thanks George, that's great! Also the UDP thing you're adding. KTorrent is moving ahead nicely indeed

I'm using KTorrent 3.3.4 and this feature still hasn't been implemented.

This most useful is to cache the write data, because the system cache write to disk immediately, which is not good for BT.

Usually, bit-torrents are not important data and do not need to be written to disk immediately. Cache them in memory and write them only when cache full (or write every 120 seconds) are much better.

I need caching badly.
For a guy having a regular download speed up to 10M/s , it is a very heavy load for the hard drive without caching data to the memory.

In net-libs/libktorrent-1.3.1, it is still not implemented. My disk is really sad..