Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ruby] Download builtin package dependencies #4473

Conversation

AndreiDreyer
Copy link
Contributor

This PR handles:

  • Scrape rubydoc to generate a NamespaceTypeMap for builtin ruby packages
  • Generate .zip folder with MessagePack files for each of the gems to be re-used
  • Write to JSON for sanity checking (to be removed)

TODO for next branch:

  • Return NamespaceTypeMap from BuiltinPackageDownloader if run
  • Implement reading from a ZipStream if the BuiltinPackageDownloader is not run and a .zip folder is present to build the NamespaceTypeMap

@max-leuthaeuser
Copy link
Contributor

I am against putting .zip files (or any binary in general) directly into this repo.
Even with Git LFS we had trouble in the past (running into the GitHub traffic limitation, any further downloads / actions disabled).

@DavidBakerEffendi
Copy link
Collaborator

@max-leuthaeuser Another solution I've thought of is to treat this like the AstGen binaries, where some repo generates these and during release-time they're compressed and packaged in a minimal way. Would that be an acceptable next step? Right now I believe these are ~100mb uncompressed

@max-leuthaeuser
Copy link
Contributor

Would that be an acceptable next step?

Guess thats much better, yes.

@DavidBakerEffendi
Copy link
Collaborator

Plan going ahead: We will not merge this PR as-is. We are going to create a separate repo that generates these type stubs, and will make releases for the language frontends that support this design (i.e., Ruby, C#)

  • Releases will be MessagePack files zipped together by language, with accompanying SHA512 checksums
  • Similar to AST GEN, there will be a download phase that will fetch the type stubs based on the frontend, validate the checksum, and unpack the archive to the resources folder

@DavidBakerEffendi
Copy link
Collaborator

This will be replaced by #4501

@DavidBakerEffendi
Copy link
Collaborator

DavidBakerEffendi commented May 1, 2024

Superseded by #4501

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants