Fix test setup to run on ARM instances

[michel-laterman]

What is the problem this PR solves?

Allow tests to run off arm platforms

How does this PR solve the problem?

Remove hard-coded amd64 values from test setup

How to test this PR locally

make test-int or make test-e2e

Related issues

• Relates E2E tests don't run on arm64, install is specific to linux #3620

[mergify]

This pull request does not have a backport label. Could you fix it @michel-laterman? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit

[mergify]

backport-8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

[swiatekm]

Speaking as someone seeing this for the first time, the fact that ARCH can be main is very surprising. Can we not use the built-in TARGETARCH and use main in the Dockerfile if the value is amd64?

[andrzej-stencel]

Or not use the built-in TARGETARCH arg at all, as it seems misused in this case. Use GO_ARCH (in line with GO_VERSION) or another arbitrary name?

[pchila]

The main arch seems a bit weird actually...
@michel-laterman Does this come from some quirk on golang-crossbuild docker tags?

[michel-laterman]

yep, it's defined in the crossbuild readme

[v1v]

Please see elastic/golang-crossbuild#446.

I recommend using the Debian-X version in the name of the Docker image tag. This might help with reproducibility.

---

That's how it's also done in Beats, see elastic/beats#34921

However in Elastic Agent -> https://github.com/elastic/elastic-agent/blob/d99b09b0769f6f34428321eedb00c0b4339c202b/dev-tools/mage/crossbuild.go#L243-L246 , I don't know the build system but it looks weird to me they use debian-8 but debian-10 in Beats.

[swiatekm]

Or not use the built-in TARGETARCH arg at all, as it seems misused in this case. Use GO_ARCH (in line with GO_VERSION) or another arbitrary name?

I wanted to use TARGETARCH, because it lets us actually build a multiplatform image. But it looks like there isn't a way to conditionally use main in the crossbuild image tag based on this. It'd be best if the crossbuild image was multiplatform as well - then this would be completely transparent, and we'd just do docker buildx build --platform=... in the Makefile.

But as this cannot be done, I'm ok with the previous approach of using a custom variable here.

Incidentally, what are the chances of making the crossbuild image multiplatform?

[pchila]

@v1v is there a specific reason why main is used instead of x86/amd64 ? I see the list of the images in golang-crossbuild README and the main arch sticks out while all the other archs follow more conventional naming (arm, mips, ppc etc.)

But at least I can see now why main is used as default arg here, to target x86/amd64 architecture

[v1v]

I believe main is a specific folder:

• https://github.com/elastic/golang-crossbuild/tree/main/go/main

See https://github.com/elastic/golang-crossbuild#gomain-docker-image

[michel-laterman]

I've changed the name variable name to SUFFIX to distance it from the TARGETARCH/ARCH concepts as it includes more information than just the architecture.
I also set it to use the same main-debian11 suffix as our other targets use.

I'm more then happy to change to TARGETARCH if the upstream sources support it.

[jlind23]

@michel-laterman I just looked at a slack conversation between @v1v and @jmlrt concluding that there was a recent change on the elasticsearch side that removed those images suffixed with the arch value.
IIUC we should just default to the multi arch one just like below:

docker pull docker.elastic.co/elasticsearch/elasticsearch:8.15.2-7f2cd899-SNAPSHOT
8.15.2-7f2cd899-SNAPSHOT: Pulling from elasticsearch/elasticsearch
b14b0d0a7610: Already exists
bc03f0deaf3c: Pull complete
9a114eab9584: Pull complete
4ca545ee6d5d: Pull complete
341b5ebe39a8: Pull complete
6b896993e0fc: Pull complete
301b398356a8: Pull complete
07428383d5cd: Pull complete
df1cde76a217: Pull complete
f7adb9f55788: Pull complete
Digest: sha256:6cbd256f19c67ee0e982064befb85b29770ec9bc71ab33d5bb623e9a2936d942
Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:8.15.2-7f2cd899-SNAPSHOT
docker.elastic.co/elasticsearch/elasticsearch:8.15.2-7f2cd899-SNAPSHOT

[mergify]

This pull request is now in conflicts. Could you fix it @michel-laterman? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b arm-test-fix upstream/arm-test-fix
git merge upstream/main
git push upstream arm-test-fix

[michel-laterman]

buildkite test this

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

[pchila]

LGTM