skipping torrents

[suxfuuu]

I dont know why but bitcannon keep skipping the torrents from kickass... can someone explain for me why?

https://kickass.to/api/

administrator@debian8:~/bitcannon_linux$ ./bitcannon_linux_386 hourlydump.txt.gz
2015/04/28 16:05:35 [OK!] Connecting to Mongo at 127.0.0.1
2015/04/28 16:05:35 [OK!] Attempting to parse
2015/04/28 16:05:35 hourlydump.txt.gz
2015/04/28 16:05:35 [OK!] File opened
2015/04/28 16:05:35 [OK!] Extension is valid
2015/04/28 16:05:35 [OK!] GZip detected, unzipping enabled
2015/04/28 16:05:35 [OK!] Reading initialized
2015/04/28 16:05:35 [OK!] Reading completed
2015/04/28 16:05:35 0 torrents imported
2015/04/28 16:05:35 8987 torrents skipped
2015/04/28 16:05:35

Press enter to quit...

[stephen304]

I haven't been able to develop during school so I'm not 100% sure, but I think skipped means it was already in the database.

Edit: I just double checked, there are several reasons for skipping:

• Already existing in the database
• Malformed file with unexpected number of columns
• Bit torrent info hash not 40 characters long
• Torrent category in blacklist

I would guess you imported an earlier dump and/or another site's dump and/or KAT's dump hasn't been updated because it hasn't been 24 hours so all of the torrents are already imported.

[stephen304]

I actually just noticed that KAT changed the format of the archive. There are now like 10 fields in the data. There isn't a quick fix, unfortunately. I wanted BitCannon to be able to learn or be flexible with different formats, but I won't be able to work on it for a week or so.

[suxfuuu]

Thank you for the feedback!

[stephen304]

No problem. When I get to it, I think I'll try to make a system that tries to apply known formats to the file, but when it fails, it lists out the fields and asks the user to identify them. Then it should save that as a new format.

I probably want to have it always double check with the first record with the user for confirmation.

[MightyRufo]

Still seem to be having this issue. Also, what kind of archives can be imported? I'm looking to import as many torrents as I possibly can. Can't import anything from kickasstorrents.

[stephen304]

Yeah sorry I haven't been able to do any work in a while - struggling to get my university to let me into computer science.

[MightyRufo]

you need more support for this from other developers and you also need more awareness. Is there anyone who can even help while you're away? I think bitcannon is a very good idea, as it allows us to have a usable local backup of the internet's torrents, while not having to worry about TPB going down again and again. Not sure why you don't have more support. With that being said. You've done a really good job so far and I like where you're going with this. Keep up the good work and good luck with your university.

Here is also a link of other archives that I would like to import: http://archiveteam.org/index.php?title=The_Pirate_Bay

Seems bitcannon can see the number of torrents, but as we know, it skips them.

[stephen304]

Thanks, I was hoping I might pique the interest of a couple of developers but that hasn't really happened. I think BC should still be usable with the other sources (other than KAT). Hopefully I'll be able to implement some kind of fix soon.

[MightyRufo]

Hopefully, but don't rush yourself. Above, I gave you a link to some other archives that I'd like to import. Maybe you can increase compatibility using these archives.

[MightyRufo]

I'm surprised this still hasn't been fixed. I realize you're busy, but you really need some help. I would really, really hate to see what you've created go to waste. If you don't plan on supporting this, what are some alternatives?

[stephen304]

Yeah it's a shame I haven't gotten anything done in a while. I'm not aware of anything close to this other than the openbay, which runs on a LAMP stack.