Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clone Bundle's URLs into lib.Archive #4532

Merged
merged 1 commit into from
Feb 10, 2025
Merged

Clone Bundle's URLs into lib.Archive #4532

merged 1 commit into from
Feb 10, 2025

Conversation

joanlopez
Copy link
Contributor

@joanlopez joanlopez commented Feb 10, 2025

What?

It modifies the Bundle.makeArchive function so Bundle's URLs (pointers) aren't directly reused by the lib.Archive it constructs, but cloned ("deep copy") and set.

Why?

Because the Archive.Write method normalizes and anonymizes these URLs to prevent certain details (like OS's user, HOME dir, etc) to be leaked to the GCk6, but we not always want these modifications to be propagated to the Bundle's URLs - e.g. when running the test script locally: k6 cloud ... --local-execution.

Review notes

A better explanation of why this (#4168) happens is because the process goes as follows:

  1. The test is loaded and configured, which includes the initialization of the first runner, and thus the cache of the module resolutions.
  2. The process of creating the test remotely also involves the creation of the archive, which modifies the URLs, as described above.
  3. Later, when the test is executed, the module resolution fails because it points to a different, non-cached, import path, due to the step 2.

Note this started to fail since we introduced this change because the old way of doing the same was k6 run -o cloud, and in that case we were not sending any archive at all. Now, the archive is an opt-out "feature", enabled by default.

Also, note that another approach to fix this issue (#4168) could consist on something like "reinstantiating" the first runner, so the "new paths" are cached again, and/or to somehow make it all work with the normalized and anonymized URLs.

However, that sounded a bit counterintuitive to me, plus I suspected that may cause other issue later (like k6 errors/warning not pointing to the correct paths locally, etc).

While doing a "deep copy" of these URLs to "isolate" changes on Archives sounded good to me, because I could not think of any scenario where that could be a problem, and logically it kinda makes sense to make the Archive "standalone".

Checklist

  • I have performed a self-review of my code.
  • I have added tests for my changes.
  • I have run linter locally (make lint) and all checks pass.
  • I have run tests locally (make tests) and all tests pass.
  • I have commented on my code, particularly in hard-to-understand areas.

Related PR(s)/Issue(s)

Fixes #4168

@joanlopez joanlopez self-assigned this Feb 10, 2025
@joanlopez joanlopez requested a review from a team as a code owner February 10, 2025 14:56
@@ -164,13 +164,16 @@ func NewBundleFromArchive(piState *lib.TestPreInitState, arc *lib.Archive) (*Bun
}

func (b *Bundle) makeArchive() *lib.Archive {
clonedSourceDataURL, _ := url.Parse(b.sourceData.URL.String())
clonedPwdURL, _ := url.Parse(b.pwd.String())

arc := &lib.Archive{
Type: "js",
Filesystems: b.filesystems,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other fields here, like this Filesystems are also passed by reference (note it's a map), and thus are likely to be modified. However, that doesn't seem to cause any issue so far.

@joanlopez joanlopez added this to the v0.57.0 milestone Feb 10, 2025
@mstoykov
Copy link
Contributor

Nice catch @joanlopez 🎉

@olegbespalov
Copy link
Contributor

Looks good! Would it be possible to introduce a test case for that (maybe in separated PR)?

@joanlopez joanlopez merged commit d649a6d into master Feb 10, 2025
28 checks passed
@joanlopez joanlopez deleted the fix-4168 branch February 10, 2025 15:44
@oleiade
Copy link
Member

oleiade commented Feb 13, 2025

Just passing by to send a huge kudos for fixing this @joanlopez 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Issue Loading Local Modules when using k6 cloud run –local-execution
4 participants