Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve automation #33

Merged
merged 7 commits into from
Dec 3, 2023
Merged

Improve automation #33

merged 7 commits into from
Dec 3, 2023

Conversation

flavio
Copy link
Member

@flavio flavio commented Nov 28, 2023

First of all, sorry about the big PR. This is something I always try to avoid, but in this case many things had to be changed at the same time to produce a stable environment.

The purpose of this PR is to set the foundation for all the refactoring/improvement work we have to do on the operator.
To achieve that, the following areas have been touched.

Code quality

  • Ensure the unit and integration tests can be run by developers on their workstations. Prior to this PR the associated make target exited with an error.
  • Enforce coding style guidelines by using golangci-lint. The linter can now be run locally using a dedicated make target
  • Ensure tests (unit, integration) and linters are run on each PR and on the main branch: this is done via GitHub actions

Automation

Prior to this PR, there was just one GitHub action that took care of building the container image and publishing the helm chart. No unit tests were involved during the whole process.

This PR removes the old GitHub action and introduces several ones.

I've tested all these changes against my personal fork. I'll share links to some examples below.

Note: these actions have been taken from the kubewarden project. This is a CNCF project I maintain. I've done some minor changes to adapted them to kwasm's needs.

Security improvement

All the GitHub actions are now referenced by their shasum. This provides a better security posture. Next to the shasum, there's a comment stating the "human" tag of the action. Dependabot can keep both information (shasum, human tag) in
sync.

Dependabot

dependabot configuration file was not correct. This prevented the bot from making PRs.

This should be fixed now and we should start receiving PRs about Dockerfile, GitHub Actions and Go dependencies updates.

Testing

As stated before, Unit and function tests, plus linters are now run on PR and each change done to the main branch.

A release won't happen if the tests are not passing. However, a PR could still be merged with broken tests; this is something we can change once this PR is merged.

You will see the tests in action against this very PR. You can see them against a smaller PR here (guess what is coming with another PR once this one gets merged 😄).

Note: the release drafter action against this PR will fail because it requires a file a configuration file that is not yet into the main branch. After this PR is merged everything will be fine.

Container image building

The container image will be built only from the contents of the main branch or with the contents referenced by a tag that follows the v* naming convention.
Prior to this commit, images were built also for PR branches, which caused issues like #31

The following tags are going to be used:

  • latest: rolling tag pointing to main
  • v<version>: a tagged release of the operator

Prior to this change the stable releases had a really long (and a bit strange IMHO) tag: :kwasm-operator-<version>.

SBOM generation

As part of the release process, SBOM files are generated for the container images (x86_64, arm64).

The SBOM files are generated using syft.

Sigstore integration

Each artifact produced by the automation pipeline (container images, SBOM files) are now signed by cosign, a tool of the Sigstore project.

Signing is done using Sigstore's keyless mode. This produces a series of OCI artifacts inside of the registry where the container image lives.

You can see them here, inside of my fork. The signatures have tags that start with sha256:<something>.

Changelog generation

Release drafter is now used to automatically build a changelog of the upcoming release.

The changelog is built by looking at the commits subjects. As a result of that, it would be great to have all the contributions follow git semantic commits guidelines. We should probably document that into the
contribution guidelines.

GitHub Release

A GitHub release will be created whenever a v* tag is pushed. The name of the GitHub Release will be v<version>.

The GitHub release will contain the information generated by the release-drafter action.

The release will also feature several artifacts like the SBOMs and the signatures of the container images (+ SBOMS). These can be used by end users to verify the integrity of all the assets we produce inside of our release pipeline.

Prior to this commit, no GitHub Release was created for the operator itself. There was just one release for the helm chart.

You can see a fake release that was created on my fork over there:

Note: the changelog isn't really fancy right now because nothing significant happened on my fork, hence release drafter didn't have that much to say.

Helm chart release

When the contents of the charts/ directory are changed, the GitHub action will create a new GitHub release called kwasm-operator-chart-<version>. This is done to differentiate it from the GitHub Release of the operator.

The action creates also a git tag named kwasm-operator-chart-<version>. Before this tag was named
kwasm-operator-<version>.

You can see examples of that on my fork:

Prior to this commit, `make test` would fail because one of the Makefile
targets was invoking `./bin/controller-gen` with the wrong arguments.

To fix the `controller-gen` invocation, the empty `config/crd/bases`
directory had to be added.

Finally, the invocation of `controller-gen` has been changed to reflect
the one generated by latest version of kube-builder.

Signed-off-by: Flavio Castelli <[email protected]>
Ensure all the tests are passing

Signed-off-by: Flavio Castelli <[email protected]>
Upgrade to Go 1.21

Signed-off-by: Flavio Castelli <[email protected]>
Introduce golangci linter

Signed-off-by: Flavio Castelli <[email protected]>
Address the warnings reported by golangci

Signed-off-by: Flavio Castelli <[email protected]>
This commit brings new automation via new GitHub actions.

Important: all the GitHub actions are now referenced by their shasum.
This provides a better security posture.
Next to the shasum, there's a comment stating the "human" tag of the
action. Dependabot can keep both information (shasum, human tag) in
sync.

Testing
=======

Unit and function tests, plus linters are now run on PR and each change done to the
`main` branch.

Container image building
============================

The container image will be built only from the contents of the `main`
branch or with the contents referenced by a tag that follows the `v*`
naming convention.
Prior to this commit, images were built also for PR branches, which
caused issues like KWasm#31

The following tags are going to be used:

  - `latest`: rolling tag pointing to `main`
  - `v<version>`: a tagged release of the operator

Prior to this change the stable releases had a really long (and a bit
strange) tag: `:kwasm-operator-<version>`.

SBOM generation
===============

As part of the release process, SBOM files are generated for the
container images (x86_64, arm64).

The SBOM files are generated using syft.

Cosign integration
==================

Each artifact produced by the automation pipeline (container images,
SBOM files) are now signed by cosign.

Signing is done using Sigstore's keyless mode.

Changelog generation
====================

Release drafter is now used to automatically build a changelog of the
upcoming release.

The changelog is built by looking at the commits subjects. As a result
of that, it would be great to have all the contributions follow git
semantic commits guidelines. We should probably document that into the
contribution guidelines.

GitHub Release
==============

A GitHub release will be created whenever a `v*` tag is pushed. The name
of the GitHub Release will be `v<version>`.

The GitHub release will contain the information generated by the
`release-drafter` action.

The release will also feature several artifacts like the SBOMs and the
signatures of the container images (+ SBOMS). These can be used by end
users to verify the integrity of all the assets we produce inside of our
release pipeline.

Prior to this commit, no GitHub Release was created for the operator
itself. There was just one release for the helm chart.

Helm chart release
==================

When the contents of the `charts/` directory are changed, the GitHub action
will create a new GitHub release called
`kwasm-operator-chart-<version>`. This is done to differentiate it from
the GitHub Release of the operator.

The action creates also a git tag named
`kwasm-operator-chart-<version>`. Before this tag was named
`kwasm-operator-<version>`.

Signed-off-by: Flavio Castelli <[email protected]>
dependabot configuration had an error which prevented its usage. Now we
track Docker, Go and GitHub Actions updates.

Signed-off-by: Flavio Castelli <[email protected]>
@shivaylamba
Copy link

Amazing work

@0xE282B0 0xE282B0 self-requested a review November 30, 2023 08:20
Copy link
Member

@0xE282B0 0xE282B0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@0xE282B0 0xE282B0 merged commit f3245d3 into KWasm:main Dec 3, 2023
2 of 3 checks passed
@0xE282B0
Copy link
Member

0xE282B0 commented Dec 3, 2023

Changes were too complex to rebase, had to squash PR.

@flavio flavio deleted the improve-automation branch December 4, 2023 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants