Hi folks,
I am just new to SunriseXP (having read the FAQ )
After AvantGo shut down, I wanted to have another possibility to have a TV guide on my Compaq iPAQ (PPC, Vade Mecum reader). I tried Plucker, but it is not able to handle cookies, which is important for
www.klack-channels.de. So the search continued...
SunriseXP did the job well at first sight. Then I discovered, that some links were not downloaded. I increased link depth and retried. Now, even less stuff is created! The pdb file, originally 220kb, is now between 2k and 50kb, it changes on every "update" event I do.
Question 1:
How come it does not update all the stuff?
Question 2:
Is there a setting "maximum file size" or something which prevents SunriseXP downloading ALL the stuff?
I'm a bit lost...
Thanks for help,
Wolferl
I've done a bit of research and found out that SunriseXP gets errors when retrieving some of the HTML files (notHandled: 403 Forbidden).
It seems there is a problem requesting the files too quickly.
Is there a setting to slow spidering down a bit? Or re-fetching the HTML if it fails?
Cheers,
Wolfgang
No, there is no setting to configure spidering speed. You might want to limit the max. number of connections to get around this. Unfortunately, this is a global setting that affects all sites.
Hi Laurens,
Quote Laurens
No, there is no setting to configure spidering speed. You might want to limit the max. number of connections to get around this.
You mean "View -> Preferences -> Network tab -> "maximum active updates"?
I already set this to "1". Unfortunately, Sunrise runs two requests at the same time then. There seems to be an "off by one" bug...
But anyways, that does not really solve the problem:
Klack-channels.de seems to block access with a "403 forbidden" if one retrieves pages too quickly.
I tried to use a special proxy software that allows throtteling (Charles Web Debugging Proxy), and using this, it was possible to get the whole thing spidered without any errors. It just takes 20 minutes (only 600kb file size)...and that proxy is not freeware.
If there would be configurable delay "delay before request" in seconds, that would be perfect.
Cheers,
Wolferl
There's not much I can do for you, unfortunately, since I'm no longer maintaining the application. You might want to look around for some other (hopefully free) proxy that can handles bandwidth throttling.
Hi Laurens,
Quote Laurens
There's not much I can do for you, unfortunately, since I'm no longer maintaining the application.
Since I have lots of programming experience...what do you think about doing this modification myself? I have searched around for the source, but wasn't successful...
Cheers,
Wolferl
Quote Wolferl
Hi Laurens,
Since I have lots of programming experience...what do you think about doing this modification myself? I have searched around for the source, but wasn't successful...
Cheers,
Wolferl
SunriseXP (C++) is unfortunately not open-source. It uses some closed-source libraries, preventing its release as an open-source project.
You might want to consider switching to the older Java version, which is open-source.
http://sourceforge.net/projects/sunrisexp/Hi Laurens,
Quote Laurens
SunriseXP (C++) is unfortunately not open-source. It uses some closed-source libraries, preventing its release as an open-source project.
Oh, too bad
Quote Laurens
You might want to consider switching to the older Java version, which is open-source.
http://sourceforge.net/projects/sunrisexp/OK, I'll stick with this one.
Thanks for your help!
Wolferl
