Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add multi arch support #28

Merged
merged 30 commits into from
Feb 8, 2024
Merged

feat: add multi arch support #28

Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
5032c57
feat: add multi arch support for java 21
Feb 6, 2024
0664dda
update 11 too
Feb 6, 2024
ca44e1f
Merge branch 'main' into multi-arch
Feb 7, 2024
0c36b58
remove gradle, use new actions
Feb 7, 2024
85a637d
Merge remote-tracking branch 'origin/main' into multi-arch
Feb 7, 2024
686e76b
Merge remote-tracking branch 'origin/multi-arch' into multi-arch
Feb 7, 2024
d85b19b
nit
Feb 7, 2024
463de30
update
Feb 7, 2024
eb876da
update
Feb 7, 2024
e855aad
checkout
Feb 7, 2024
30e298c
try again
Feb 7, 2024
2cd826b
try again
Feb 7, 2024
67cb346
fix dep
Feb 7, 2024
f6b0cfc
blunder
Feb 7, 2024
e035f4a
check
Feb 7, 2024
8515f63
check
Feb 7, 2024
1609699
delimit
Feb 7, 2024
483b25b
test
Feb 7, 2024
e7b7fab
test
Feb 7, 2024
27e6eeb
test
Feb 7, 2024
b709112
test
Feb 7, 2024
5138652
final
Feb 7, 2024
2c4dfe9
load
Feb 7, 2024
f386295
change name
Feb 7, 2024
f510a92
load
Feb 7, 2024
5f2f680
use matrix as cannot load multiplatform
Feb 7, 2024
d503643
update
Feb 8, 2024
2efd76f
bug
Feb 8, 2024
51836e6
bug
Feb 8, 2024
0e64d58
bug
Feb 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/extract-args.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
on:
workflow_call:
inputs:
file:
required: true
type: string
image:
required: true
type: string
outputs:
tags:
description: "Image tags with all java versions appended to image input"
value: ${{ jobs.parse.outputs.tags }}
args:
description: "Build args content read from input file"
value: ${{ jobs.parse.outputs.args }}

jobs:
parse:
runs-on: ubuntu-22.04
outputs:
tags: ${{ steps.extract.outputs.tags }}
args: ${{ steps.extract.outputs.args }}
steps:
- name: Checkout
uses: actions/checkout@v4

- shell: bash
id: extract
run: |
TOTAL_JAVA_VERSION=$(cat ${{ inputs.file }} | grep "JAVA_VERSION" | cut -d'=' -f2)
BASE_JAVA_VERSION=$(echo $TOTAL_JAVA_VERSION | cut -d'-' -f1)
MAJOR_JAVA_VERSION=$(echo $BASE_JAVA_VERSION | cut -d'.' -f1)
echo "tags=${{ inputs.image }}:$MAJOR_JAVA_VERSION,${{ inputs.image }}:$BASE_JAVA_VERSION,${{ inputs.image }}:$TOTAL_JAVA_VERSION" >> $GITHUB_OUTPUT
{
echo "args<<nEOFn"
echo "$(cat ${{ inputs.file }})"
Copy link
Author

@kishansairam9 kishansairam9 Feb 7, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wasted too much time here because of not wrapping this in echo not creating new line as it is not in file and leading to EOF not found 😔

echo "nEOFn"
} >> $GITHUB_OUTPUT
62 changes: 48 additions & 14 deletions .github/workflows/pr-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,43 +3,77 @@ on:
push:
branches:
- main
pull_request_target:
branches:
- main
pull_request:

jobs:
extract-args-java-11:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like excessive complexity for a simple task. In the build case, we don't even need to generate all the individual tags anyway. In the publish version, since that's the only spot we need them, you can just inline the couple cut calls.

Copy link
Author

@kishansairam9 kishansairam9 Feb 7, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is a bit complex, but since we wouldn't want to store build args hidden in github actions yamls rather in repo source I thought this is required

Can we inline reading data from a file and doing cut to pass as a variable to an action? I couldn't think of a way to do it and built a reusable workflow. I reused the same in build too even if isn't strictly required

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you do something like this? basically just an inline form of the same code. I also moved the build arg name from inside the file to outside.

      - name: Read Java Version
        shell: bash
        id: versions
        run: |
          ZULU_JDK_VERSION=`cat ${{ matrix.java-major-version}}/java.version}}`
          echo ZULU_JDK_VERSION=${ZULU_JDK_VERSION} >> $GITHUB_OUTPUT
          echo JAVA_MAJOR_VERSION=$(echo $ZULU_JDK_VERSION | cut -d'.' -f1)" >> $GITHUB_OUTPUT
          echo JAVA_FULL_VERSION=$(echo $ZULU_JDK_VERSION | cut -d'-' -f1) >> $GITHUB_OUTPUT

Then, for build args just declare it like

build-args: |
                      JAVA_VERSION=${{steps.versions.output.ZULU_JDK_VERSION}}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also wouldn't bother publishing the zulu version tag - it should be treated as an impl detail since we're not making any guarantees about the specific distro of java we're using.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine by me, I wanted to leave scope for any more build args being added in future without need to touch gha

Not a strong opinion, but would prefer to have the current args.env file version as:

  1. it is not too complex to understand from actions perspective

  2. decent tradeoff of complexity in a gha yaml file which will be rarely looked upon for the benefit of not visiting gha part when someone adds a new build arg (which I agree will be rare, but so is someone trying to understand this reusable workflow)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why we'd add any more args, and if we did, it's likely we'd be changing the build too. If the arg was statically defined, there'd be no point in passing it in in the first place. If it were dynamic, we'd likely be using a matrix or some other way of calculating the value for it in CI then need to pass them in like above anyway.

The java version is actually bit of a strange case as it's static, but kept outside the dockerfile so it can be consumed to tag as well. A common counter example would be something like the current commit SHA.

So in other words, the more likely future case is we'd need to calculate a build arg in the build and then pass it to the build command which would be more difficult to do if merging it with the static env file. Not positive if that makes sense, or if I'm missing your point though.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The java version is actually bit of a strange case as it's static, but kept outside the dockerfile so it can be consumed to tag as well.

Ah, got it. I thought it was done as a good practice whatever things are not required to be static, make them as args and pass it during build. Hence I was thinking of multiple build args as a possibility.

Makes sense to me now, updating

uses: ./.github/workflows/extract-args.yml
with:
file: java-11/args.env
image: hypertrace/java-build

extract-args-java-21:
uses: ./.github/workflows/extract-args.yml
with:
file: java-21/args.env
image: hypertrace/java-build

build:
runs-on: ubuntu-22.04
strategy:
matrix:
aaron-steinfeld marked this conversation as resolved.
Show resolved Hide resolved
platform: [linux/amd64, linux/arm64]
needs:
- extract-args-java-11
- extract-args-java-21
steps:
# Set fetch-depth: 0 to fetch commit history and tags for use in version calculation
- name: Check out code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Checkout
uses: actions/checkout@v4

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_READ_USER }}
password: ${{ secrets.DOCKERHUB_READ_TOKEN }}

- name: Build with Gradle
uses: hypertrace/github-actions/gradle@main
- name: Build java-11
uses: docker/build-push-action@v5
with:
context: java-11
platforms: ${{ matrix.platform }}
build-args: |
${{ needs.extract-args-java-11.outputs.args }}
tags: ${{ needs.extract-args-java-11.outputs.tags }}
load: true

- name: Build java-21
uses: docker/build-push-action@v5
with:
args: dockerBuildImages
context: java-21
platforms: ${{ matrix.platform }}
build-args: |
${{ needs.extract-args-java-21.outputs.args }}
tags: ${{ needs.extract-args-java-21.outputs.tags }}
load: true

- name: Scan java-11 image
uses: hypertrace/github-actions/trivy-image-scan@main
with:
image: hypertrace/java
image: hypertrace/java-build
tag: 11
category: java-11
output-mode: github

- name: Scan java-21 image
uses: hypertrace/github-actions/trivy-image-scan@main
with:
image: hypertrace/java
image: hypertrace/java-build
tag: 21
category: java-21
output-mode: github
60 changes: 45 additions & 15 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,25 +7,55 @@ on:
workflow_dispatch:

jobs:
publish-artifacts:
extract-args-java-11:
uses: ./.github/workflows/extract-args.yml
with:
file: java-11/args.env
image: hypertrace/java

extract-args-java-21:
uses: ./.github/workflows/extract-args.yml
with:
file: java-21/args.env
image: hypertrace/java

publish:
runs-on: ubuntu-22.04
needs:
- extract-args-java-11
- extract-args-java-21
steps:
# Set fetch-depth: 0 to fetch commit history and tags for use in version calculation
- name: Check out code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Checkout
uses: actions/checkout@v4

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Login to Docker Hub
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_READ_USER }}
password: ${{ secrets.DOCKERHUB_READ_TOKEN }}
username: ${{ secrets.DOCKERHUB_PUBLISH_USER }}
password: ${{ secrets.DOCKERHUB_PUBLISH_TOKEN }}

- name: publish docker image
uses: hypertrace/github-actions/gradle@main
- name: Publish java 11
uses: docker/build-push-action@v5
with:
context: java-11
platforms: linux/amd64,linux/arm64
build-args: |
${{ needs.extract-args-java-11.outputs.args }}
tags: ${{ needs.extract-args-java-11.outputs.tags }}
push: true

- name: Publish java 21
uses: docker/build-push-action@v5
with:
args: dockerPushImages
env:
DOCKER_USERNAME: ${{ secrets.DOCKERHUB_PUBLISH_USER }}
DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_PUBLISH_TOKEN }}
context: java-21
platforms: linux/amd64,linux/arm64
build-args: |
${{ needs.extract-args-java-21.outputs.args }}
tags: ${{ needs.extract-args-java-21.outputs.tags }}
push: true
3 changes: 0 additions & 3 deletions build.gradle.kts
View file
Open in desktop

This file was deleted.

5 changes: 0 additions & 5 deletions gradle.properties
View file
Open in desktop

This file was deleted.

Binary file removed gradle/wrapper/gradle-wrapper.jar
View file
Open in desktop
Binary file not shown.
7 changes: 0 additions & 7 deletions gradle/wrapper/gradle-wrapper.properties
View file
Open in desktop

This file was deleted.

Loading
Loading