I'm not sure if this is done, or even can be done... but I'm hoping.

When seeding a torrent, giving preference to chunks that no leecher has (at least, no leecher that is currently connected to me - can't tell about leechers that aren't connected to me) would be sweet.

I don't really know anything about the torrent chunk negotiation, but perhaps if other clients are asking for a bunch of chunks, and ktorrent is figuring out which ones to honour immediately, giving preference to clients requesting chunks that no one has would make sense.

It'd allow me to stop seeding earlier, I think As in, rather than giving chunk #100 to everyone in the swarm, I give it to one person who can then start swarming it out, meanwhile giving out a new chunk to someone (else?) who can swarm it out when I'm done.

Of course, while leeching, this preference doesn't need to be involved (although it can), and once all chunks have been given out (or, rather, between the connected leechers, they have all the chunks), then it doesn't matter, either.

Tanktalus wrote:I'm not sure if this is done, or even can be done... but I'm hoping.

When seeding a torrent, giving preference to chunks that no leecher has (at least, no leecher that is currently connected to me - can't tell about leechers that aren't connected to me) would be sweet.

I don't really know anything about the torrent chunk negotiation, but perhaps if other clients are asking for a bunch of chunks, and ktorrent is figuring out which ones to honour immediately, giving preference to clients requesting chunks that no one has would make sense.


I do not know how far the fast extension protocol is implemented in KTorrent... But in theory, if all clients support it, you can suggest pieces to another peer.

It'd allow me to stop seeding earlier, I think As in, rather than giving chunk #100 to everyone in the swarm, I give it to one person who can then start swarming it out, meanwhile giving out a new chunk to someone (else?) who can swarm it out when I'm done.

Well, it can only work if all clients implement the fast extension protocol.
Let's see what George thinks about this.

stoeptegel wrote:I do not know how far the fast extension protocol is implemented in KTorrent... But in theory, if all clients support it, you can suggest pieces to another peer.

That's interesting, though it goes much farther than this suggestion. In the fast extension, a peer may suggest what they think is the best thing for a client to do. My suggestion was less forceful. Even without the fast extension, merely honouring "good" requests before honouring "poor" requests would basically passively do something similar. Clients requesting "good" (unique) choices would get their chunks sooner, poor choices would be slower. Not slower as in bandwidth, but slower as in, if I run out of upload slots before answering your request, well, too bad.

stoeptegel wrote:Well, it can only work if all clients implement the fast extension protocol.
Let's see what George thinks about this.

The goal of my suggestion is to attempt to lower the amount I need to seed before a swarm can start taking care of itself. Once that point is reached, there will become no "good" choice (as there won't be any chunks left that I'm the sole purveyor of), and then all requests are on equal footing again (well, subject to the rest of ktorrent's algorithms).

Of course, once someone gets all the pieces, s/he will become a seeder, and ktorrent (smartly) disconnects from them, which may mean that, of all the peers currently connected, there may be chunks that no one has (just me and the other seeder(s)), and the algorithm kicks in again.

This should also promote the ability of each peer to start sharing as much as possible (because the first time through, they'll all have chunks that no one else has other than the seeder, so they can start uploading to anyone), and that sharing is likely to start happening sooner.

This should, at least according to my theory, start getting peers to share sooner, get everyone to have the full torrent sooner (since there's more sharing going on), get everyone to a 1:1 ratio sooner (assuming they participate nicely at all), and reduce the bandwidth required by seeders (since more leechers will have more to share).

Of course, the next step in this would be that the fewer peers with a requested chunk would increase that chunk's importance in sharing. That is, if 0 peers have a chunk, anyone requesting it gets the top priority, if 1 peer has that chunk, the request gets the next level of priority (i.e., after anyone asking for 0-peer chunks), if 2 peers have that chunk the request is ignored until and unless all 0-peer and 1-peer requests are answered (or are being answered - upload slot given), etc. But this would require not a bitmask for all the chunks but a full count (probably a byte or two - or four/eight if you use natural int's) for each chunk, which would take up way more space and CPU time to calculate.

Tanktalus wrote:

stoeptegel wrote:I do not know how far the fast extension protocol is implemented in KTorrent... But in theory, if all clients support it, you can suggest pieces to another peer.

That's interesting, though it goes much farther than this suggestion. In the fast extension, a peer may suggest what they think is the best thing for a client to do. My suggestion was less forceful. Even without the fast extension, merely honouring "good" requests before honouring "poor" requests would basically passively do something similar. Clients requesting "good" (unique) choices would get their chunks sooner, poor choices would be slower. Not slower as in bandwidth, but slower as in, if I run out of upload slots before answering your request, well, too bad.

Interesting proposal. Let' s see what george thinks about your fast-extenstion-first-seat proposal.

The goal of my suggestion is to attempt to lower the amount I need to seed before a swarm can start taking care of itself. Once that point is reached, there will become no "good" choice (as there won't be any chunks left that I'm the sole purveyor of), and then all requests are on equal footing again (well, subject to the rest of ktorrent's algorithms).

I agree on this, there is only a need for suggest when there is only one partial seed in the swarm

Of course, once someone gets all the pieces, s/he will become a seeder, and ktorrent (smartly) disconnects from them, which may mean that, of all the peers currently connected, there may be chunks that no one has (just me and the other seeder(s)), and the algorithm kicks in again.

Well, i dunno how this works with lazy bitfield and when certain clients do not send have messages as a feature.(like bitrocket)

This should also promote the ability of each peer to start sharing as much as possible (because the first time through, they'll all have chunks that no one else has other than the seeder, so they can start uploading to anyone), and that sharing is likely to start happening sooner.

This should, at least according to my theory, start getting peers to share sooner, get everyone to have the full torrent sooner (since there's more sharing going on), get everyone to a 1:1 ratio sooner (assuming they participate nicely at all), and reduce the bandwidth required by seeders (since more leechers will have more to share).

Of course, the next step in this would be that the fewer peers with a requested chunk would increase that chunk's importance in sharing. That is, if 0 peers have a chunk, anyone requesting it gets the top priority, if 1 peer has that chunk, the request gets the next level of priority (i.e., after anyone asking for 0-peer chunks), if 2 peers have that chunk the request is ignored until and unless all 0-peer and 1-peer requests are answered (or are being answered - upload slot given), etc. But this would require not a bitmask for all the chunks but a full count (probably a byte or two - or four/eight if you use natural int's) for each chunk, which would take up way more space and CPU time to calculate.

Sorry, my time run out, i will read it later...

Tanktalus wrote:I'm not sure if this is done, or even can be done... but I'm hoping.

When seeding a torrent, giving preference to chunks that no leecher has (at least, no leecher that is currently connected to me - can't tell about leechers that aren't connected to me) would be sweet.

I don't really know anything about the torrent chunk negotiation, but perhaps if other clients are asking for a bunch of chunks, and ktorrent is figuring out which ones to honour immediately, giving preference to clients requesting chunks that no one has would make sense.

Most clients ask the rarest chunks first, so I don't think this is really needed.

It'd allow me to stop seeding earlier, I think As in, rather than giving chunk #100 to everyone in the swarm, I give it to one person who can then start swarming it out, meanwhile giving out a new chunk to someone (else?) who can swarm it out when I'm done.

So in essence you want superseeding, which is a rather ugly hack of the bittorrent protocol (and this is mostly the biggest reason why we have not yet implemented this, it would probably result in some ugly hackery in our code).

George wrote:

Tanktalus wrote:I'm not sure if this is done, or even can be done... but I'm hoping.

When seeding a torrent, giving preference to chunks that no leecher has (at least, no leecher that is currently connected to me - can't tell about leechers that aren't connected to me) would be sweet.

I don't really know anything about the torrent chunk negotiation, but perhaps if other clients are asking for a bunch of chunks, and ktorrent is figuring out which ones to honour immediately, giving preference to clients requesting chunks that no one has would make sense.

Most clients ask the rarest chunks first, so I don't think this is really needed.

Well, that's not entirely true. Clients can ask to focus on a single file (e.g., the first track in a CD, or the first CD or DVD in a set). Which is fine ... but when all of the first leeches do the same thing, it does mean I spend far more time / bandwidth seeding than I should need to. This proposal would allow ktorrent to focus on those who aren't asking for just the first track / CD / DVD / image / whatever, instead allowing those people to feed each other while I focus on those who are leeching the entire torrent.

This will somewhat reduce the effectiveness of trying to only leech a single part of the torrent, or even just trying to prioritise chunks. But not hugely - as they can still get those chunks from otherwise-less-busy peers (i.e., other leechers). That is, they should still get their desired pieces faster than if they wanted the entire torrent with equal priority, so there'd still be advantage in doing what they're doing, but not as much advantage as there is now, though the difference is likely to be fairly minimal, if not outright negligible. In exchange, there'd be a significant advantage for the torrent as a whole, especially as more and more clients may start using this type of system.

George wrote:

Tanktalus wrote:It'd allow me to stop seeding earlier, I think As in, rather than giving chunk #100 to everyone in the swarm, I give it to one person who can then start swarming it out, meanwhile giving out a new chunk to someone (else?) who can swarm it out when I'm done.

So in essence you want superseeding, which is a rather ugly hack of the bittorrent protocol (and this is mostly the biggest reason why we have not yet implemented this, it would probably result in some ugly hackery in our code).

See, *there* is, IMNSHO, a good reason not to do it. "[U]gly hackery" is a perfectly valid reason to avoid it. While the concept is, I think, a benefit to the protocol, if it results in less-maintainable code whereby we wouldn't get other features as quickly (or at all), well, that's not good.

Well, I can see the reasoning behind this and personally as a torrent user I would also prefer giving seeding priority to those clients and transfers that are requesting files or chunks that are less available than others, i.e. as a torrent user I am primarily interested in content availability, that is I want to ensure that I -and others- can get complete downloads/contents easily, and thus I would be more than willing to improve content availability by giving priority to data segments that are comparatively 'rare', while postponing transfers of files/chunks that are highly available.
Which might ultimately even enable other users to stop seeding because their data has been made "sufficiently" available.

All the best,
Segmented