Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bundler: Provide the build configuration #654

Conversation

a-ovchinnikov
Copy link
Collaborator

In order for Bundler to build from the content prefetched by Cachi2, we need to set a few configuration parameters. There are several ways of doing this, but the most robust one is to place a local configuration file into processed repository (see docs/design/bundler.md for details).

This commit introduces a fetch step that presets configuraion parameters.

Maintainers will complete the following section

  • Commit messages are descriptive enough
  • Code coverage from testing does not decrease and new code is covered
  • Docs updated (if applicable)
  • Docs links in the code are still valid (if docs were updated)

Note: if the contribution is external (not from an organization member), the CI
pipeline will not run automatically. After verifying that the CI is safe to run:

cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
@a-ovchinnikov a-ovchinnikov marked this pull request as ready for review September 20, 2024 12:38
@slimreaper35 slimreaper35 mentioned this pull request Sep 23, 2024
4 tasks
Copy link
Member

@eskultety eskultety left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since unlike Yarn v1 none of this bundler work is tracked as upstream issues, I'm missing the part where the config is actually

Edit: Since I eventually provided all comments inline, please ignore ^this snippet I forgot to delete before submitting the review.

cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
@a-ovchinnikov a-ovchinnikov force-pushed the bundler_provide_build_config branch 2 times, most recently from c7764b3 to b1e2183 Compare September 30, 2024 23:22
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
Comment on lines +56 to +83
pytest.param(
utils.TestParameters(
repo="https://github.com/cachito-testing/cachi2-bundler.git",
ref="well_formed_ruby_all_features",
packages=({"path": ".", "type": "bundler"},),
flags=["--dev-package-managers"],
check_output=False,
check_deps_checksums=False,
check_vendor_checksums=False,
expected_exit_code=0,
expected_output="",
),
id="bundler_everything_present",
),
pytest.param(
utils.TestParameters(
repo="https://github.com/cachito-testing/cachi2-bundler.git",
ref="well_formed_ruby_without_gemspec",
packages=({"path": ".", "type": "bundler"},),
flags=["--dev-package-managers"],
check_output=False,
check_deps_checksums=False,
check_vendor_checksums=False,
expected_exit_code=0,
expected_output="",
),
id="bundler_everything_present_except_gemspec",
),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth adding a scenario with multiple packages in the same repo ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not, at least not with our current ITs structure. This scenario would essentially exercise the same code that is exercised in existing tests, but in a loop.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last thing. We don't have any e2e tests and I believe we could make one of those tests above (that are almost identical) to do full pre-fetch and build an image.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I'll add a e2e test in a separate commit to this PR (after it passes locally).

cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
tests/unit/package_managers/bundler/test_main.py Outdated Show resolved Hide resolved
cachi2/core/package_managers/bundler/main.py Outdated Show resolved Hide resolved
tests/unit/package_managers/bundler/test_main.py Outdated Show resolved Hide resolved
tests/unit/package_managers/bundler/test_main.py Outdated Show resolved Hide resolved
In order for Bundler to build from the content prefetched by Cachi2, we
need to set a few configuration parameters. There are several ways of
doing this, creating an alternative config and pointing bundler to it
via an environment variable was chosen as the least of all evils.
The resulting config file will override all settings defined in
all configs committed to all repositories within the package.
An attempt is made to remediate the issue by cloning top-level
config first and adding necessary values to it, but this could fail
on a complex package which contains sub-packages and nested
local configs. Such cases should be very rare and this risk is deemed
acceptable.

Signed-off-by: Alexey Ovchinnikov <[email protected]>
Copy link
Member

@eskultety eskultety left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e2e isn't a deal breaker for me at this point given there may be more work on the backend needed, so it'll have to be added by the time we flip the support switch on it. The rest of the changes look good -> ACK.

@a-ovchinnikov a-ovchinnikov added this pull request to the merge queue Oct 3, 2024
Merged via the queue into containerbuildsystem:main with commit 93915e7 Oct 3, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants