Mobileread
Progress Sync across different platforms
#1  ilovejedd 05-12-2019, 06:23 PM
Finally got around to installing KOReader on my Kobos (Aura HD and Glo HD).

So far so good except for progress sync. Syncing works fine between PW3 & PW4 and between Aura HD & Glo HD. Alas, syncing between Kobo & Kindle isn't working.

How does progress sync work? The wiki states the following so I assumed it was going by checksum or something. Given it won't sync progress on identical files between Kobo & Kindle, it appears it's using some other identifier that's not intrinsic to the epub. The epubs even have the same path and filename except for obvious reasons the Kindle's on /mnt/us/ and the Kobo's on /mnt/onboard/.
Quote
This plugin does not send file name or any file content to the koreader sync server. And all data transferred between the "Progress sync" plugin and the sync server are secured by HTTPS (Hypertext Transfer Protocol Secure) connections.
Reply 

#2  NiLuJe 05-12-2019, 09:00 PM
There's possibly a bit of hashing involved, but take that with a grain of salt, as I'm extremely unfamiliar with that bit of code.

Are the actual files identical on your end? How do you transfer them? (Might be a case of Calibre using a different output template?).
Reply 

#3  ilovejedd 05-13-2019, 12:40 AM
Quote NiLuJe
There's possibly a bit of hashing involved, but take that with a grain of salt, as I'm extremely unfamiliar with that bit of code.

Are the actual files identical on your end? How do you transfer them? (Might be a case of Calibre using a different output template?).
Files are bit for bit identical. I was able to trace the digest function used.

Spoiler Warning below






Code
-- calculate partial digest of the document and store in its docsettings to avoid document saving
-- feature to change its checksum.
--
-- To the calculating mechanism itself.
-- since only PDF documents could be modified by KOReader by appending data
-- at the end of the files when highlighting, we use a non-even sampling
-- algorithm which samples with larger weight at file head and much smaller
-- weight at file tail, thus reduces the probability that appended data may change
-- the digest value.
-- Note that if PDF file size is around 1024, 4096, 16384, 65536, 262144
-- 1048576, 4194304, 16777216, 67108864, 268435456 or 1073741824, appending data
-- by highlighting in KOReader may change the digest value.
function Document:fastDigest(docsettings) if not self.file then return end local file = io.open(self.file, 'rb') if file then local tmp_docsettings = false if not docsettings then -- if not provided, open/create it docsettings = require("docsettings"):open(self.file) tmp_docsettings = true end local result = docsettings:readSetting("partial_md5_checksum") if not result then logger.dbg("computing and storing partial_md5_checksum") local bit = require("bit") local md5 = require("ffi/MD5") local lshift = bit.lshift local step, size = 1024, 1024 local m = md5.new() for i = -1, 10 do file:seek("set", lshift(step, 2*i)) local sample = file:read(size) if sample then m:update(sample) else break end end result = m:sum() docsettings:saveSetting("partial_md5_checksum", result) end if tmp_docsettings then docsettings:close() end file:close() return result end
end


Looks like KOSync uses the partial_md5_checksum as identifier for syncing. My problem was I just overwrote fanfics with updated versions leading to wrong partial_md5_checksum in metadata.epub.lua.

I deleted the sidecar folder forcing KOReader to recalculate the md5 sum and sync worked.
Reply 

Today's Posts | Search this Thread | Login | Register