Code 503 while syncing highlights with readwise

[sharmarkeGit]

Hello,

I have an issue with syncing my highlights with readwise from a Kobo Aura Edition 2 v1.

I get this error message:
"There was a problem sending your highlights: received a non-200 status code from Readwise: code 503"

I can see in the logs that the highlights have been parsed:
"highlight_count":12036,"level":"info","msg":"Successfully parsed highlights"

Can you please help me?

Regards

---

System Details

• Version: v1.3.0
• Platform: darwin
• Architecture: amd64

[marcus-crane]

Hi sharmarkeGit,

Hmm, a 503 here would indicate that the issue has occurred on Readwise's side (versus a 4xx code which indicates an application issue)

It looks like you used the new issue button under Settings. If you open Settings again, you should see a button to open the application logs in Finder.

Could you get the latest logs (it should have today's date) and email them to me at [email protected]?

From there, I can try and see what might have gone wrong and if needed, I can try to replicate the issue.

It could be that the number of highlights is too large for Readwise to ingest. There isn't an explicitly mentioned limit but I might imagine that 12000 is near the limit hahaha

[sharmarkeGit]

Thanks for your prompt reply!
I've sent you the logs.

In case the problem is the number of highlights, is there any way to only select highlights for one book or a few books at a time?

[marcus-crane]

Thanks for your prompt reply! I've sent you the logs.

In case the problem is the number of highlights, is there any way to only select highlights for one book or a few books at a time?

What October should do is just make multiple requests one after the other so I'll have to do some testing to figure out a reasonable batch size and then I can issue an updated version that splits highlights if the request is going to be bigger than whatever the server can handle.

I'll ask the Readwise team if they can give any hints on what seems like a reasonable maximum as well

[marcus-crane]

@sharmarkeGit Heya, I've just released a new beta build which you can find on the releases page. If you're not sure, just pick the darwin-universal build which works on both Intel and Apple Silicon.

It splits up highlights into batches of 2000 (I just picked a random number until I hear from the Readwise team) so let me know if that solves you issue and I'll roll it out to everyone else.

Also, some other questions you asked were

is there any way to only select highlights for one book or a few books at a time?

There isn't but I'd like to provide something like that (#45). Ideally, each book would be submitted individually (but in a parallel) so failure of highlights in one book don't affect uploading unrelated highlights.

An example might be if your Kobo bugged out while highlight and highlighted 20 pages (although Readwise would still batch that up and send it as that issue came up once)

The other part of making things more efficient would be tracking which highlights have been seen (#16) and storing that information on your Kobo (just reading it as a USB basically) so that you don't need to upload 12000 highlights every time (which Readwise deduplicates so nothing happens if you send them over and over)

[sharmarkeGit]

Duuude!!! It works!!!!!!

You are absolutely amazing, thank you so much! It didn't even take a day. This is some Outstanding support

[marcus-crane]

You're welcome!

Thankfully the batching part of the code has some tests which made it easy to know I hadn't broken anything 😅

I'll roll out the updated version later today although you can feel free to skip the update given it'll be the exact same as the current beta you've got now.