Sync with Github error (Request failed with status code: 504)

[joshmoore]

This is a re-opening of #957. Please see the discussion there.

Issue is (additionally) related to the number of organizations and repositories a user is a member of.

The only workaround currently is to disconnect repositories and reconnect them, which for users with many repositories is very sub-optimal.

Related to Zenodo Support email [Ticket#332699]

[joshmoore]

On the Zenodo Support email thread:

We are aware of an issue with the Github integration for users that have a large amount of repositories, such as in your case. Unfortunately, we haven't fixed this yet and we still need to investigate the issue further to solve it. What happens is that we iterate over the list of repositories that we fetch from the user's Github account, which causes the request from the client to eventually timeout and return an HTTP 504 Gateway Timeout.
Sometimes the sync process manages to complete, even if the request timed out, but it is unclear to us how to force this. For instance, I tried to sync your repositories and couldn't make it work.

In the future, we might have a workaround for this, but we can't commit to a date/timeframe since our team is fully committed to working on other features.

We are sorry for the inconvenience this is causing you.

[krokicki]

I'm also consistently getting this error. It is a 504 timeout after 30 seconds. I only have one organization, but it's a large one: https://github.com/JaneliaSciComp

This effectively means that we can no longer use Zenodo for new repositories, which is really a shame.

To anyone from Zenodo reading this: this type of error is usually easy to mitigate by increasing the read timeout of your reverse proxy. For example, with Nginx the relevant configuration option is proxy_read_timeout.

[bethac07]

+1 for this; I'm a member of a number of large orgs and this would really help!

[caufieldjh]

I am also experiencing this error and I'm in ~16 different orgs.

[lldelisle]

I am also experiencing this error...

[alin256]

Having the 504 issue today. Tried clicking sync several times within an hour without success.

Does anyone know the server "napping" to awake time (assuming this is a temporary problem)?

[alin256]

Aha - for me logging out and in again (in Zenodo) resulted in a successful sync.

[VladimirAlexiev]

I get the same problem when trying to publish a W3C CG effort: w3c-cg/awesome-semantic-shapes#37 .

• I have 125 repos listed at https://zenodo.org/account/settings/github/ (coming from 3 orgs), and another org has allowed OAuth from Zenodo
• Now each time I sync, I get error 504 .
• I logged out and when I logged in, it said "(updated 10 min ago)", even though the last update failed.
• Trying sync again gets error 504

[joshmoore]

Thanks everyone for the continuing reports!! Also a big thumbs up to listing your number of orgs and repositories to provide data points for the maintainers.

[joshmoore]

Ack!! I clicked on the wrong comment button. 🤦🏽 Can maintainer please re-open? Apologies for the noise.

cc: @slint @alejandromumo @zzacharo @yashlamba