Whenever I notice that a user controlled download is stalled, I stop the download, and then start it again, and this will give me download rates of anywhere from 40-60kB/s. but this rate will slowly decrease, until the download is stalled again (usually after 10 seconds). If I stop and start the torrent again, it will do the same thing. So if I want to get a download really fast, I have to manually stop and restart each torrent every 10 seconds (no I'm not actually doing this, I'm just saying.....).

The choking algorithm will favor new peers to get them started, so this is probably what you are seeing. You can do a manual announce to get some fresh blood.