[Flaky Test]: TestStandaloneUpgradeRetryDownload – Error: Failed trigger upgrade of daemon: open : The system cannot find the file specified.

[pchila]

Failing test case

TestStandaloneUpgradeRetryDownload

Error message

Error: Failed trigger upgrade of daemon: open : The system cannot find the file specified.

Build

https://buildkite.com/elastic/elastic-agent-extended-testing-bk/builds/1350#0194daae-65c2-4d84-83d3-99c13e0c7e13

OS

Windows

Stacktrace and notes

The agent tries to download the archive twice before giving up (the requests are denied both times on purpose as we can see from the test logs), hence the failure

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[pchila]

Looking at the test logs the agent retries download correctly but then there's an issue with extraction

{"log.level":"error","@timestamp":"2025-02-06T01:24:57.517Z","log.origin":{"function":"github.com/elastic/elastic-agent/internal/pkg/agent/application/upgrade.(*Upgrader).unpack","file.name":"upgrade/step_unpack.go","file.line":51},"message":"Failed to unpack upgrade artifact","log":{"source":"elastic-agent"},"error":{"message":"open : The system cannot find the file specified."},"version":"8.18.0-SNAPSHOT","file.path":"C:\\Program Files\\Elastic\\Agent\\data\\elastic-agent-9.0.0-SNAPSHOT-345f2a\\downloads\\elastic-agent-8.18.0-SNAPSHOT-windows-x86_64.zip","unpack_result":{"hash":"","versioned-home":""},"ecs.version":"1.6.0"}

This looks similar to #6729 and #6732

[pchila]

Managed to reproduce the issue on a windows 2022 VM.
The issue is coming from this line

elastic-agent/internal/pkg/agent/install/flavors.go

Line 56 in b59d51a

f, err := os.Open(registryPath)

called from here

elastic-agent/internal/pkg/agent/application/upgrade/step_unpack.go

Line 110 in b59d51a

skipFn, err := skipFnFromZip(log, r, flavor, fileNamePrefix, createVersionedHomeFromHash(hash), registry)

with an uninitialized (nil) map as registry.

This lines have been added with PR #6542

@michalpristas could please have a look here? It seems that downgrading from 9.0.0 to 8.18.0 does not work as expected.

[cmacknz]

This looks like a real bug for "backwards" upgrades, how did #6542 merge with this?

Am I blind or did the integration tests not actually run for the final commit to that PR that let it merge? https://buildkite.com/elastic/elastic-agent/builds/16075#0194ac78-496d-4ddb-874b-f8b0f3c72634

I see a step to trigger them but the triggered run isn't linked anywhere.