It appears you are correct, will fix it tomorrow.

George wrote:It appears you are correct, will fix it tomorrow.

Ok, just found another place that should be fixed to process torrents > 4G

It is possible to create torrents >4Gb but just try to check their data integrity
and it will fail due to following:

libktorrent/datachecker/singledatachecker.cpp
...
fptr.seek(File::BEGIN,i*tor.getChunkSize());
...

for torrents >4Gb it will seek max at 2048 so casting to Uint64 is extremely needed

--
Alexey

ranger2k wrote:

George wrote:It appears you are correct, will fix it tomorrow.

...

for torrents >4Gb it will seek max at 2048 so casting to Uint64 is extremely needed

--
Alexey

Ok, just realized that there are several places where you didn't cast result
of expressions to Uint64 and all these places should be fixed to handle files
>4Gb

--
Alexey

ranger2k wrote:
Ok, just realized that there are several places where you didn't cast result
of expressions to Uint64 and all these places should be fixed to handle files
>4Gb

--
Alexey

Ok, finally after fixing all these places I was able to create/seed/check integrity
of torrent > 4Gb.

Also I noted that when I am trying to download torrent and point to the dir where requested file already exists ktorrent didn't start checking integrity automatically
as it was before(2.0.3). Did I miss something and this didn't work on my side only?

ranger2k wrote:

ranger2k wrote:
Ok, just realized that there are several places where you didn't cast result
of expressions to Uint64 and all these places should be fixed to handle files
>4Gb

--
Alexey

Ok, finally after fixing all these places I was able to create/seed/check integrity
of torrent > 4Gb.

Also I noted that when I am trying to download torrent and point to the dir where requested file already exists ktorrent didn't start checking integrity automatically
as it was before(2.0.3). Did I miss something and this didn't work on my side only?

So any chance to fix all these ::seek(expression) calls in order to cast expressions to Uint64 in svn?

see patch below:

Index: libktorrent/torrent/dndfile.cpp
===================================================================
--- libktorrent/torrent/dndfile.cpp (revision 626165)
+++ libktorrent/torrent/dndfile.cpp (working copy)
@@ -160,7 +160,7 @@
if (hdr.last_size + off > buf_size)
return 0;

- fptr.seek(File::BEGIN,sizeof(DNDFileHeader) + hdr.first_size);
+ fptr.seek(File::BEGIN,(Uint64)(sizeof(DNDFileHeader) + hdr.first_size));
return fptr.read(buf + off,hdr.last_size);
}

@@ -199,7 +199,7 @@

// put everything in tmp buf
memcpy(tmp,buf,hdr.first_size);
- fptr.seek(File::BEGIN,sizeof(DNDFileHeader) + hdr.first_size);
+ fptr.seek(File::BEGIN,(Uint64)(sizeof(DNDFileHeader) + hdr.first_size));
fptr.read(tmp + hdr.first_size,hdr.last_size);

// update the hash of the header
Index: libktorrent/torrent/torrentcreator.cpp
===================================================================
--- libktorrent/torrent/torrentcreator.cpp (revision 626165)
+++ libktorrent/torrent/torrentcreator.cpp (working copy)
@@ -71,7 +71,7 @@
last_size = tot_size % chunk_size;
Out() << "Tot Size : " << tot_size << endl;

- tot_size = fi.size();
+// tot_size = fi.size();
}

if (last_size == 0)
@@ -242,7 +242,7 @@
.arg(target).arg(fptr.errorString()));

Uint32 s = cur_chunk != num_chunks - 1 ? chunk_size : last_size;
- fptr.seek(File::BEGIN,cur_chunk*chunk_size);
+ fptr.seek(File::BEGIN,(Uint64)cur_chunk*chunk_size);

fptr.read(buf,s);
SHA1Hash h = SHA1Hash::generate(buf,s);
Index: libktorrent/datachecker/singledatachecker.cpp
===================================================================
--- libktorrent/datachecker/singledatachecker.cpp (revision 626165)
+++ libktorrent/datachecker/singledatachecker.cpp (working copy)
@@ -80,7 +80,7 @@
// read the chunk
Uint32 size = i == num_chunks - 1 && tor.getFileLength() % tor.getChunkSize() >
tor.getFileLength() % tor.getChunkSize() : (Uint32)tor.getChunkS
- fptr.seek(File::BEGIN,i*tor.getChunkSize());
+ fptr.seek(File::BEGIN,(Uint64)i*tor.getChunkSize());
fptr.read(buf,size);
// generate and test hash
SHA1Hash h = SHA1Hash::generate(buf,size);
Index: apps/ktcachecheck/singlecachechecker.cpp
===================================================================
--- apps/ktcachecheck/singlecachechecker.cpp (revision 626165)
+++ apps/ktcachecheck/singlecachechecker.cpp (working copy)
@@ -63,7 +63,7 @@
{
Uint32 size = i == num_chunks - 1 && tor.getFileLength() % tor.getChunkSize() >
tor.getFileLength() % tor.getChunkSize() : (Uint32)tor.getChunkS
- fptr.seek(File::BEGIN,i*tor.getChunkSize());
+ fptr.seek(File::BEGIN,(Uint64)(i*tor.getChunkSize()));
fptr.read(buf,size);
SHA1Hash h = SHA1Hash::generate(buf,size);
bool ok = (h == tor.getHash(i));

The parameters to the seek function is an Int64, so I didn't see much need in casting it to an unsignet 64 bit integer. Further more in most of these cases you are already multiplying with a 64 bit unsigned integer, the compiler should handle this.

George wrote:The parameters to the seek function is an Int64, so I didn't see much need in casting it to an unsignet 64 bit integer. Further more in most of these cases you are already multiplying with a 64 bit unsigned integer, the compiler should handle this.

In other words - you are saying that you are able to create/check integrity/seed torrents >4 Gb with the the latest svn version of ktorrent?

From my observation without cast second arg to Unit64 in seek() especially in singledatachecker.cpp I was not able to check integrity of torrent > 4G.

--
Alexey

I will test it out, note that this line in torrentcreator.cpp :

tot_size = fi.size();

Shouldn't have been there, it seems we forgot to remove that. (It is removed in the latest SVN version)

EDIT: first I need to find a 4+ GB file

George wrote:I will test it out, note that this line in torrentcreator.cpp :

tot_size = fi.size();

Shouldn't have been there, it seems we forgot to remove that. (It is removed in the latest SVN version)

EDIT: first I need to find a 4+ GB file

as you can see above I've also commented out this line as I also believe that it was wrong assignment.

regarding 4Gb files try below oneliner from previous page of this thread:

touch test.file ; perl -e 'truncate("test.file", 4*1024*1024*1024+100);'

OK, you are right, still don't understand why this happens.

Anyway it is fixed.

I still have problem checking integrity of successfully created torrents with size >4gb
I`m using ktorrent 2.1, opensuse 10.2 x86_64, xfs, kernel 2.6.18.5-181

first i thought it`s a kernel bug
http://www.kernel.org/git/?p=linux/kern ... 48431222b3
but later i found this topic..

waiting for reply

The last time I tested it (after I fixed the bug), it worked, I even verified it with azureus to be sure.

Are you sure you are running 2.1 ?

Yes i am. I downloaded 2.1 right from here:
http://ftp.gwdg.de/pub/opensuse/reposit ... x86_64.rpm

Creating torrent >4gb with azureus (bittorrent, bittornado etc) and checking integrity of this torrent in ktorrent works fine (but i dont like azureus, you see :)

Stil i didnt trying creating torrent with ktorrent and checking its integrity in azureus. Should i?

I think tarick has the same problem:

P.S. I think, I found one other bug, but I test it again. Yesterday I failed to create new torrent with large files (total size approx 9GB, chunk size 4MB). After creation I remove torrent and add it again - ktorrent mark all chunks after 1024 as bad or not downloaded. So, I create torrent with uTorrent1.6 and ktorrent add it correctly.

Could it be x86_64 issue? Or maybe something wrong with torrent creation routine?

nick31 wrote:Yes i am. I downloaded 2.1 right from here:
http://ftp.gwdg.de/pub/opensuse/reposit ... x86_64.rpm

Creating torrent >4gb with azureus (bittorrent, bittornado etc) and checking integrity of this torrent in ktorrent works fine (but i dont like azureus, you see

Stil i didnt trying creating torrent with ktorrent and checking its integrity in azureus. Should i?

What kind of files did you put in the torrent ? Are we talking single file or multi file ? If it is a multi file torrent, how big where the files ?

Could it be x86_64 issue? Or maybe something wrong with torrent creation routine?

There has to be a bug in there somewhere. I'll do some tests tomorrow, if I can find the time.

George wrote:What kind of files did you put in the torrent ? Are we talking single file or multi file ? If it is a multi file torrent, how big where the files ?

Just tried to create multifile torrent (total size of all files in the torrent was about 8gb, mp3 directory, sizes of files vary from 3 to 150mb). Integrity check was ok.

So its a problem of a single file with size >4gb, i guess.