Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix test setup to run on ARM instances #3904

Merged
merged 5 commits into from
Sep 23, 2024
Merged

Fix test setup to run on ARM instances #3904

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
cc15fe3
Fix test setup to run on ARM instances
michel-laterman Sep 13, 2024
aa6b460
Change arg to TARGETARCH
michel-laterman Sep 13, 2024
2e9427f
Merge branch 'main' into arm-test-fix
michel-laterman Sep 19, 2024
608a7cc
Change DOCKERARCH to SUFFIX, change to main-debian11
michel-laterman Sep 19, 2024
fb415f5
Merge branch 'main' into arm-test-fix
michel-laterman Sep 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Dockerfile.build
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
ARG GO_VERSION
FROM docker.elastic.co/beats-dev/golang-crossbuild:${GO_VERSION}-main-debian11
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speaking as someone seeing this for the first time, the fact that ARCH can be main is very surprising. Can we not use the built-in TARGETARCH and use main in the Dockerfile if the value is amd64?

Copy link

@andrzej-stencel andrzej-stencel Sep 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or not use the built-in TARGETARCH arg at all, as it seems misused in this case. Use GO_ARCH (in line with GO_VERSION) or another arbitrary name?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main arch seems a bit weird actually...
@michel-laterman Does this come from some quirk on golang-crossbuild docker tags?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, it's defined in the crossbuild readme

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see elastic/golang-crossbuild#446.

I recommend using the Debian-X version in the name of the Docker image tag. This might help with reproducibility.

That's how it's also done in Beats, see elastic/beats#34921

However in Elastic Agent -> https://github.com/elastic/elastic-agent/blob/d99b09b0769f6f34428321eedb00c0b4339c202b/dev-tools/mage/crossbuild.go#L243-L246 , I don't know the build system but it looks weird to me they use debian-8 but debian-10 in Beats.

Copy link

@swiatekm swiatekm Sep 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or not use the built-in TARGETARCH arg at all, as it seems misused in this case. Use GO_ARCH (in line with GO_VERSION) or another arbitrary name?

I wanted to use TARGETARCH, because it lets us actually build a multiplatform image. But it looks like there isn't a way to conditionally use main in the crossbuild image tag based on this. It'd be best if the crossbuild image was multiplatform as well - then this would be completely transparent, and we'd just do docker buildx build --platform=... in the Makefile.

But as this cannot be done, I'm ok with the previous approach of using a custom variable here.

Incidentally, what are the chances of making the crossbuild image multiplatform?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@v1v is there a specific reason why main is used instead of x86/amd64 ? I see the list of the images in golang-crossbuild README and the main arch sticks out while all the other archs follow more conventional naming (arm, mips, ppc etc.)

But at least I can see now why main is used as default arg here, to target x86/amd64 architecture

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe main is a specific folder:

See https://github.com/elastic/golang-crossbuild#gomain-docker-image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the name variable name to SUFFIX to distance it from the TARGETARCH/ARCH concepts as it includes more information than just the architecture.
I also set it to use the same main-debian11 suffix as our other targets use.

I'm more then happy to change to TARGETARCH if the upstream sources support it.

ARG TARGETARCH # Should be main or arm
FROM docker.elastic.co/beats-dev/golang-crossbuild:${GO_VERSION}-${TARGETARCH}

RUN \
apt-get update \
Expand Down
12 changes: 8 additions & 4 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@ endif
DOCKER_IMAGE_TAG?=${VERSION}
DOCKER_IMAGE?=docker.elastic.co/fleet-server/fleet-server


PLATFORM_TARGETS=$(addprefix release-, $(PLATFORMS))
COVER_TARGETS=$(addprefix cover-, $(PLATFORMS))
COMMIT=$(shell git rev-parse --short HEAD)
Expand Down Expand Up @@ -248,7 +247,12 @@ else
endif

build-releaser: ## - Build a Docker image to run make package including all build tools
docker build -t $(BUILDER_IMAGE) -f Dockerfile.build --build-arg GO_VERSION=$(GO_VERSION) .
ifeq ($(shell uname -p),arm)
$(eval DOCKERARCH := arm)
else
$(eval DOCKERARCH := main)
endif
docker build -t $(BUILDER_IMAGE) -f Dockerfile.build --build-arg GO_VERSION=$(GO_VERSION) --build-arg TARGETARCH=${DOCKERARCH} .

.PHONY: docker-release
docker-release: build-releaser ## - Builds a release for all platforms in a dockerised environment
Expand All @@ -257,7 +261,7 @@ docker-release: build-releaser ## - Builds a release for all platforms in a dock
.PHONY: docker-cover-e2e-binaries
docker-cover-e2e-binaries: build-releaser
## Build for local architecture and for linux/amd64 for docker images.
docker run --rm -u $(shell id -u):$(shell id -g) --volume $(PWD):/go/src/github.com/elastic/fleet-server -e SNAPSHOT=true $(BUILDER_IMAGE) cover-linux/amd64 cover-$(shell go env GOOS)/$(shell go env GOARCH)
docker run --rm -u $(shell id -u):$(shell id -g) --volume $(PWD):/go/src/github.com/elastic/fleet-server -e SNAPSHOT=true $(BUILDER_IMAGE) cover-linux/$(shell go env GOARCH) cover-$(shell go env GOOS)/$(shell go env GOARCH)

.PHONY: release
release: $(PLATFORM_TARGETS) ## - Builds a release. Specify exact platform with PLATFORMS env.
Expand Down Expand Up @@ -336,7 +340,7 @@ test-int-set: ## - Run integration tests without setup
.PHONY: build-e2e-agent-image
build-e2e-agent-image: docker-cover-e2e-binaries ## - Build a custom elastic-agent image with fleet-server binaries with coverage enabled injected
@printf "${CMD_COLOR_ON} Creating test e2e agent image\n${CMD_COLOR_OFF}"
GOARCH=amd64 ./dev-tools/e2e/build.sh
./dev-tools/e2e/build.sh

.PHONY: e2e-certs
e2e-certs: ## - Use openssl to create a CA, encrypted private key, and signed fleet-server cert testing purposes
Expand Down
4 changes: 2 additions & 2 deletions dev-tools/e2e/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
services:
kibana:
image: "docker.elastic.co/kibana/kibana:${ELASTICSEARCH_VERSION}-amd64"
image: "docker.elastic.co/kibana/kibana:${ELASTICSEARCH_VERSION}"
container_name: kibana
healthcheck:
test: ["CMD", "curl", "-f", "${ELASTICSEARCH_USERNAME}:${ELASTICSEARCH_PASSWORD}@localhost:5601/api/status"]
Expand All @@ -17,7 +17,7 @@ services:
networks:
- integration
apm-server:
image: "docker.elastic.co/apm/apm-server:${ELASTICSEARCH_VERSION}-amd64"
image: "docker.elastic.co/apm/apm-server:${ELASTICSEARCH_VERSION}"
container_name: apm-server
# curl is not in the apm-server image
#healthcheck:
Expand Down
10 changes: 4 additions & 6 deletions dev-tools/integration/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ volumes:
driver: local
services:
setup:
image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION}-amd64
image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION}
volumes:
- certs:/usr/share/elasticsearch/config/certs
user: "0"
Expand Down Expand Up @@ -45,7 +45,7 @@ services:
find . -type f -exec chmod 640 \{\} \;;
';
elasticsearch:
image: "docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION}-amd64"
image: "docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION}"
container_name: elasticsearch
environment:
- node.name=es01
Expand Down Expand Up @@ -84,12 +84,11 @@ services:
- certs:/usr/share/elasticsearch/config/certs
ports:
- 127.0.0.1:9200:9200

elasticsearch-remote:
depends_on:
setup:
condition: service_healthy
image: "docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION}-amd64"
condition: service_completed_successfully
image: "docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION}"
container_name: elasticsearch-remote
environment:
- node.name=es02
Expand Down Expand Up @@ -127,4 +126,3 @@ services:
- certs:/usr/share/elasticsearch/config/certs
ports:
- 127.0.0.1:9201:9200

Loading