Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vendoring: Do shallow clones per default #1025

Open
mss opened this issue Feb 5, 2025 · 1 comment · May be fixed by #984
Open

Vendoring: Do shallow clones per default #1025

mss opened this issue Feb 5, 2025 · 1 comment · May be fixed by #984

Comments

@mss
Copy link
Contributor

mss commented Feb 5, 2025

Describe the Feature

This came up in a discussion on #1019: Doing shallow Git clones can speed up vendoring extremely. Since vendoring will throw away the Git history anyway there is no point in pulling in the whole history in the first place.

Expected Behavior

atmos vendor pull could be quicker for Git repos.

Use Case

Improve people's life by improving the speed of atmos vendor pull per default.

Describe Ideal Solution

  1. Check each Git vendor URL if the query parameter depth was specified.
    2a. If no such parameter was given (eg. git::https://github.com/cloudposse/atmos.git?ref=v1.158.0) add depth=1 to the query parameters (ie. git::https://github.com/cloudposse/atmos.git?ref=v1.158.0&depth=1).
    2b. If such a parameter was given and the value is empty (eg. just git::https://github.com/cloudposse/atmos.git?depth=&ref=v1.158.0) treat this as an override to pull the whole history and remove that empty parameter (ie. git::https://github.com/cloudposse/atmos.git?ref=v1.158.0).
    2c. Otherwise: Just keep the depth which was probably specified for some good reason (eg. git::https://github.com/cloudposse/atmos.git?depth=10&ref=v1.158.0)

One thing to watch out for: I am not sure if shallow clones work without a ref. So maybe the existence of the ref query parameter has to be checked in step 1 as well.

Alternatives Considered

Just don't do this.

Additional Context

We use vendoring excessively. Here are some totally unscientific number of our default config (shallow clones via with depth=1) and with full clones:

# rm -rf components/terraform/
# time atmos vendor pull | wc -l
54

real    0m30,154s
user    0m3,870s
sys     0m3,282s
# sed -i -e 's/depth=1&//' vendor/*.yaml
# rm -rf components/terraform/
# time atmos vendor pull | wc -l
54

real    1m12,948s
user    0m8,421s
sys     0m4,575s

@GabisCampana
Copy link

@osterman @aknysh

@osterman osterman linked a pull request Feb 7, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants