Hi All,

I've just commit an rssfeed plugin for ktorrent to svn.

I'd appreciate any testing people are willing to do.

Feel free to mail bugs to me - skyphyr@gmail.com

Cheers,

Alan.

I'll test it ASAP. I just need to check out Georges trackers patch a bit and I'll post the results here.

I'm sure other people will do too, so expect some feedback very soon.

Okay, I tried it a bit and here's what I came up with.

I'm not quite sure I understand how this works so I'm gonna explain what I did:
I typed RSS title and URL, and IMHO it lacks 'fetch' button or something like that, but pressing enter seems to have done the trick. BTW, what does 'Active' checkbox means? Also, could you explain 'Autorefresh' and 'Ignore TTL' a bit? How does it work?

RSS I've tried:
http://thepiratebay.org/rss

http://www.scenereactor.org/rssdl.php
and none of them seem to work, or at least they don't with KT.

RSS from got listed properly but KT crashed when I tried to download some torrent. Backtrace is sent to your email so you can take a look. It seems to me that XML parsing got stuck somewhere but knowing that you're still working on that I'm hoping this backtrace could help.

So far, I would suggest a 'Fetch' button and some feedback so that the user knows when KT tries to fetch rss (unlike with mentioned rss feeds where I don't know what's happening since there's no GUI notification).

BTW, it would be nice if someone would left more RSS feed URLs so we could have more of them to test.

Hi Ivan,

Thanks for the links - I'll check them out tonight.

At the moment it automatically checks the feed when change signals are received from any of the properties - I'll disconnect most of them and shift it to the active being toggled on and a fetch button. hmmm actually I might have put a refresh button in there which I just need to hook up.

I've tested it with the torrentspy rss feeds - so I'd be interested to here if you get the same issues with those.

Cheers,

Alan.

Hi Ivan,

I think some of this is UI issues that I'll need to deal with.

I've successfully got thepiratebay and demonoid feeds to list. Downloading works fine on thepiratebay one here, but because the demonoid one requires login it's not working as it just hands a page without any torrents when the link is requested. I'm not sure how to get it working as I'm guessing it requires cookies and username/password details.

I'll get the refresh button hooked up so you can use it to manually load a feed after filling in the URL. The active toggle is required to download the feed. When it's toggled on the feed is refreshed and the timer starter to refresh it. I tend to put the URL in then click active. Also if the title is set to New or is empty then it will get the title from the feed.

Rss feeds have a ttl property - which is how often the site hosting the feed would like you to refresh it. When ignoreTTL is off the rssfeed plugin will use the ttl for the frequency at which it should refresh the feed. If you switch ignoreTTL on then you can set the refresh frequency yourself.

The filters tab has a few more options. There are two lists of filters which you create - accept filters and reject filters. The reject filters are always run first - anything which is found in reject filters won't be processed in accept filters (i.e. won't be downloaded). This way you can add filters to prevent any vcd, cam etc from being downloaded.

The regular expressions is a list of regular expressions for the current filter. Every expression in the list (the listedit used for entry doesn't matter) needs to match for the filter to be considered a match. The main reason I've made a list instead of a single regular expression is it make it easier for newbs to add a list of words instead of complex regexs. It also means some things can be added and order won't matter - i.e. cool.*movie could be one and xvid could be another. So it won't matter if it appears in the feed as "Cool Movie Xvid" or "Xvid Cool Movie". Actually with that particular regex set "Cool Xvid Movie" would also match.

The treat as series tells it whether to check for an episode number - it also means it won't download the same episode twice. If you put Download Matches Without Episode then if it can't find an episode number it will download it anyway - if this is disabled (and treat as series is on) there has to be an episode number or it won't be a match. Any values for season or episode set to 0 will match anything. So a min season of 1 and a max season of 2 with 0 for episodes will download all episodes in season 1 and 2. Also if you set min to season 1 episode 5 and max to season 3 episode 14 then all of season two will match - not just season 2 episode 5-14.

I'll hook up the test text stuff now - this will be able to test any text you put in there against the filter - when the lineedit's background turns green it's a match and red if it isn't - it'll be white when the text hasn't been tested.

I'll go take care of those issues now and look into why some other feeds aren't loading.

Cheers,

Alan.

Hi,

Another update - http://tvrss.net/feed/vtv/ was listed as one of the ones with no articles - this was because of the maximum time to keep articles defaults to 3 days - this date is relative to the publication date rather than the time the feed is read in. I've changed the default to 30 days as a little voice inside me says it should be based on the age of the article, not when ktorrent first read it, but if the court of public opinion deems that it should be date of first reading then I'll change it.

Cheers,

Alan.

A whole stack of updates and fixes are in svn now.

You can now manually download a feed.
Testing of filters works.
You can trigger reprocessing of a filter (though I haven't yet tested that because of issues here).
General niceness improvements with enabling/disabling of entry forms when there isn't an active feed/filter.

If you can give details on any feeds/articles which cause crashes that'd be good as I haven't managed to get repro on that yet.

And of course send any other issues etc my way.

Cheers,

Alan.

Me again... still hogging the thread.

I've put it more updates. I've completely rewritten the download code so it processes html pages by hand. This should get rid of the issue causing the crash as it's not even linking to the library which was wiping it out before.

It should also support more feeds now as if it can't figure out their date it uses the time it was downloaded instead.

Once again bug reports (or functionality reports if it actually works for you) to me please

Cheers,

Alan.

Look pretty advanced. I am not a huge rss user besides some easy akregator stuff, so i might not be of big help. Maybe i should work some more with it to get the hang of it.

I think it would be slick if it could also load the "read more"(akregator style naming) rss weblinks into the dafault system browser, besides the download links we now have. (not in KTorrent because of lazy pwmanager usage)

Bugs i noticed:
- (re)Naming a filter comes with having to re-focus the location after every keyboard hit.
- Sometimes KTorrent says(translated from my own translation..) : File or folder /var/tmp/kdecache-stoeptegel/krun/xxxxxx.torrent doesn't exists. This is not related to rss though, seeing that this also happens when openen a torrent with ktorrent from konqueror (KTorrent already running).

Thanks stoeptegel.

I had the refocussing issue once, but it stopped here - I'll try figure out what caused it and get that sorted.

EDIT: Got the refocusing issue fixed - could you grab fresh source and check it's sorted please?

I'm not sure that message relates to the rssfeed stuff, but I'll check to make sure.

Cheers,

Alan.

Just to let you all know there's another update.

It's fixed crashes on checking for bittorrent (some html pages would cause it to crash).

I've also improved the download handling code so it now works with way more pages. Let me know of any feeds (which don't require you to login) that aren't working. I'm not sure how we'd go about handling pages that require login as it depends on how each of those work.

You should also get a popup now if it fails to track down a torrent from the link when you click download.

Cheers,

Alan.

clicked cancel on a torrent download, it crashed ktorrent. Not to mention that when first creating a filter, even if you click "active" it isn't actually active, and clicking "process" just does nothing. Not realising thats what it was, I restarted ktorrent, and on going back to the rss tab, the new filter was marked as inactive. enabling it (again) caused it to fetch the latest item.

edit: seems it only crashed once, not sure what caused it, maybe it was mininova taking too damn long.

edit2: But for some reason, its trying to get the mininova /get link twice? Or maybe more, the second one never finishes, stupid mininova.

edit3: After leaving the download/copy window open for several minutes, pressing cancel causes ktorrent to crash. Is there a way you can set the timeout lower, and then retry later on?

Also, using kio and its silly dialogs is rater obtrusive. it blocks all input to ktorrent.

IMO ktorrent, and its rss feed plugin should make all long running tasks, like file checking and torrent download/copy's happen in the background, with a nice little status meter in the torrents info.

Hi Tomasu,

Thanks for that - I was using standard kde functions for downloads and it seems making it wait is a bit bodgy (I'm surprised how many sites are actually slow). The multiple downloads thing will be because they don't always link directly to torrent files - many feeds link to a html file - which the plugin then has to process to check for which link is the actual torrent.

I'm going to rewrite the download processing with a new class that handles all the work itself - so it'll be able to use signals and slots and leave everything nice and snappy - ideally I'll even find something which doesn't pop up a downloading dialog.

Cheers,

Alan.

If you'd like help, I'd be happy to. I've got a handle on Qt

Yay for double posting.

Also, I have a little bug report, seems some torrents on demonoid are parsed oddly.

http://www.strangesoft.net/ktorrent-rss ... monoid.png

the season and episode numbers are a little off.

Also, is there a way you can make the link part copyable? so we can just copy it and open a browser and paste it? Or even linkable?

edit: links that are a little off: http://www.demonoid.com/files/details/458698/775222
http://www.demonoid.com/files/details/458160/775222
The numbers in the rss pane seem to match the first set in the links here.