Mobileread
v0.1.2 very slow when editing on Mac
#1  DanielCoffey 08-15-2009, 10:27 AM
Valloric - if you are already aware of this problem just let me know but if not, I'll raise an issue for you...

=====

Sigil v0.1.2
Mac OS Leopard 10.5.8
iMac 24" 4Gb Ram

ISSUE : Sigil is very slow to open The_Deed_of_Paksenarrion.epub from Baen Webscriptions, taking around 90 seconds. The epub is 1.4Mb in size and contains around 30-ish chapters (1350 pages or so on my Sony 505).

It takes between 8-10 seconds to respond to a single character deletion and around 15 seconds to respond to a carriage return in the middle of a paragraph, displaying the Mac rotating beachball each time.

In addition, it opens this epub and displays it centered whereas my Sony 505 displays the epub as left justified... I not sure if this is just the default behaviour where justification is not set in the document.

Let me know if you want me to raise an official issue (and include the epub).

Cheers,

Daniel.
Reply 

#2  Valloric 08-15-2009, 10:40 AM
Quote DanielCoffey
ISSUE : Sigil is very slow to open The_Deed_of_Paksenarrion.epub from Baen Webscriptions, taking around 90 seconds. The epub is 1.4Mb in size and contains around 30-ish chapters (1350 pages or so on my Sony 505).

It takes between 8-10 seconds to respond to a single character deletion and around 15 seconds to respond to a carriage return in the middle of a paragraph, displaying the Mac rotating beachball each time.
There have been some reports of issues like this. From what I can gather, it is very rare.

Tell me, does this happen with all epub books, or just this one? If it's just this one, then create the issue and attach the epub... otherwise I don't know how to help you. If it happens with all epub books, then it must be your system.

Quote DanielCoffey
In addition, it opens this epub and displays it centered whereas my Sony 505 displays the epub as left justified... I not sure if this is just the default behaviour where justification is not set in the document.
This is unrelated, and not a Sigil issue, but a Sony PRS-505 issue. The Mobile Digital Editions version on the 505's always displays books left justified, even if the markup in them specifies full justification. So your book is probably fully justified.

They fixed this in later versions of Mobile DE, hopefully we'll get it in the firmware update that's coming at the end of this month.
Reply 

#3  DanielCoffey 08-15-2009, 11:16 AM
Thanks for the quick reply...

Oddly enough, another smaller epub I have does it too, but to a much lesser degree. War For The Oaks epub (free from Tor) is 368Kb and the delays are smaller - around 10-12 seconds to open and only slightly sluggish responses when editing.

===

System : iMac 2.8GHz Intel Core 2 Duo, 4Gb RAM
OS : Mac OS X 10.5.8

Sigil idle, no ebook loaded, 0% CPU, 25Mb mem used

When opening War For The Oaks (368Kb), Sigil's CPU usage goes to 100% and its memory jumps from 25Mb used to 45Mb used then CPU usage settles at 0% once epub is loaded.

When entering carriage returns in the middle of a paragraph, CPU usage is about 25% but when deleting them, CPU usage hits 90-100% and it takes around 1s per deletion.

I then closed Sigil and reopened it before loading the 1.4Mb Deed of Paksenarrion epub. Activity monitor settled at 100% CPU usage and quickly reported Sigil (Not Responding) and memory usage climbed rapidly to 112.5Mb used. Once the epub was loaded, CPU settled back to 0%.

I am not sure if that degree of memory and CPU usage is what you would expect.
Reply 

#4  Valloric 08-15-2009, 12:55 PM
Quote DanielCoffey
Oddly enough, another smaller epub I have does it too, but to a much lesser degree.
So what you're saying is that you have epubs for which this doesn't happen? Then attach the epub for which it does so I can take a look at what's going on.

Quote DanielCoffey
When opening War For The Oaks (368Kb), Sigil's CPU usage goes to 100% and its memory jumps from 25Mb used to 45Mb used then CPU usage settles at 0% once epub is loaded.
This is normal if it takes a few seconds. The loading is CPU intensive because of tidyLib which reads in your HTML and rewrites it completely.

Quote DanielCoffey
When entering carriage returns in the middle of a paragraph, CPU usage is about 25% but when deleting them, CPU usage hits 90-100% and it takes around 1s per deletion.
On my machine (Core 2 Duo, 4GB RAM), so far I have yet to see an editing task take more than 5% of CPU. But I've tried deleting paragraphs as you've noted, and this does seem to jump the usage to 100% for God knows what reason... but only on HUGE, 50k XHTML lines-of-code files. It's 2% CPU and instant response for average novel-length files (on my machine).

When editing huge files, there's going to be some unavoidable slowdown for Book View. It's still an embedded webkit-based browser.

Quote DanielCoffey
I then closed Sigil and reopened it before loading the 1.4Mb Deed of Paksenarrion epub. Activity monitor settled at 100% CPU usage and quickly reported Sigil (Not Responding) and memory usage climbed rapidly to 112.5Mb used. Once the epub was loaded, CPU settled back to 0%.

I am not sure if that degree of memory and CPU usage is what you would expect.
As I've said, loading is intensive. There's probably room for optimization there.

Even for huge files, memory usage should remain well below 100MB, between 40 and 70 MB. The Peak Memory Set has for me gone beyond 100MB (102MB actually), but that's for a split second during loading. It always goes down afterwards.

As reported, Sigil should be able to handle very large files even on weak hardware.

This is all on Windows though. I haven't had the chance to do performance testing for Macs much. From my limited testing with an old Macbook, it seems fine though.

But 90-second loading and several-second lag for typing? That should never happen. Attach that epub so I can take a look at it. If you're worried about your file being made available on the internet, just add the "Private" label to the issue which makes it visible only to the issue reporter (you) and the project developers (me).

Without the epub files in question I can only guess. If you provide me with it, and the same slowdown happens on my system, then I can tell you what the problem is (and hopefully I can fix it). If it doesn't happen on my system with the same file, then it must be your system that's at fault.
Reply 

#5  DanielCoffey 08-15-2009, 01:33 PM
I have created private issue 82 for you with the slow epub attached. I hope it behaves the same on your PC as it does on mine.

I must say that Calibre and Sigil make a wonderful pair of applications. You should be truly proud of all your hard work!
Reply 

#6  Valloric 08-15-2009, 02:18 PM
Quote DanielCoffey
I have created private issue 82 for you with the slow epub attached. I hope it behaves the same on your PC as it does on mine.
It does. It most certainly does.

It just that Sigil is not the problem. The epub file is. The markup is... horrible, to say the least. For instance, every paragraph with text starts like this:
Code
<p onmouseover="PNo(1032)">
And there is no javascript included in the book, from what I can tell. This only serves to slow down browsers. Also, between every two paragraphs with text there is code like this:
Code
<p><a id="p1033" name="p1033"/></p>
Which does God knows what.

And I see what you mean when you said the text was centered. I thought you meant fully justified, but no, it is really centered. Removing the CSS style that applies "text-align: center" fixed this.

With that CSS style gone, and after removing all the useless "p/anchors" and the onmouseover handlers with some regexes in notepad++, your file now takes up 30MB less memory and can be nicely edited in Sigil with no lag. So CPU usage solved.

The memory consumption is still around 100MB, but your file is 80k XHTML lines-of-code. It is 1.4MB as an epub because epubs are compressed ZIP archives, and plain text gets compressed nicely. In Sigil, text is in uncompressed UTF-16 which means at least two bytes for every character, whereas your file is English in UTF-8, so only one byte per character stored. This effectively doubles the memory required to store your text in Sigil.

Now take into account that because of technical limitations of Qt widgets used for Code View and Book View, there are three text buffers instead of one, and that Book View is an embedded web browser... these things add up.

In the end it was your file that was causing the lag. Must have been an old version of calibre they were using when they created it, because that's some really painful markup.
Reply 

#7  DanielCoffey 08-15-2009, 02:40 PM
Fantastic! Er, I mean fantastic that it was reproducible, of course. I will feed your comments back to the owner of the Webscriptions site and get my copy cleaned up.

Thanks for looking at this and apologies for it taking time away from your other work and fun. Please feel free to close the issue (82).

As I said - well done for working on Sigil (even the logo is gorgeous!).
Reply 

#8  Dave Berk 08-18-2009, 05:47 AM
I also have the same problem. With one file. So it's probably a problem with the file itself. But opening the file with a text editor or an html editor is no problem and editing is fast then too.

It's the same file I attached with issues 87 & 88.
Reply 

#9  Valloric 08-18-2009, 10:29 AM
Quote Dave Berk
I also have the same problem. With one file. So it's probably a problem with the file itself. But opening the file with a text editor or an html editor is no problem and editing is fast then too.

It's the same file I attached with issues 87 & 88.
I'm guessing these other editors are not rendering out all of the XHTML files as one flow.

I'm also starting to believe the whole "one flow for editing" idea may not be so amazing.
Reply 

#10  Dave Berk 08-18-2009, 12:23 PM
Quote Valloric
I'm guessing these other editors are not rendering out all of the XHTML files as one flow.
Yep.

Quote Valloric
I'm also starting to believe the whole "one flow for editing" idea may not be so amazing.
What other option is there?
Reply 

  Next »  Last »  (1/2)
Today's Posts | Search this Thread | Login | Register