Skip to content

Commit

Permalink
Changes to Jenkinsfile to no longer publish to NPM
Browse files Browse the repository at this point in the history
Also, remove the determine-release-type.sh file. This script is now embedded in the Jenkinsfile.

Some changes to deploy/jenkins-ci/README.md were done, but it's not ready yet.
  • Loading branch information
israel-hdez committed May 7, 2021
1 parent 1ee129e commit 5e0b5bb
Show file tree
Hide file tree
Showing 3 changed files with 107 additions and 176 deletions.
176 changes: 104 additions & 72 deletions deploy/jenkins-ci/Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,9 @@
* - QUAY_OPERATOR_NAME
* defaultValue: quay.io/kiali/kiali-operator
* description: The name of the Quay repository to push the operator release
* - NPM_DRY_RUN
* - SKIP_KIALI_SERVER_RELEASE
* defaultValue: n
* description: Set to "y" if you want to make a "dry run" of the front-end release process
* - SKIP_BACKEND_RELEASE
* defaultValue: n
* description: Set to 'y' if you don't want to release the backend
* - SKIP_UI_RELEASE
* defaultValue: n
* description: Set to 'y' if you don't want to release the UI
* description: Set to 'y' if you don't want to release the server (back-end and front-end)
* - SKIP_OPERATOR_RELEASE
* defaultValue: n
* description: Set to 'y' if you don't want to release the operator
Expand All @@ -50,10 +44,6 @@
* - SKIP_SITE_RELEASE
* defaultValue: n
* description: Set to 'y' if you don't want to release the website
* - UI_VERSION
* defaultValue: ''
* description: If you are skipping UI release. Specify the UI version to package, or leave
* unset to use the version present in the main Makefile (e.g. leave unset for patch releases)
* - NPM_CONFIG_REGISTRY
* defaultValue: ''
* description: NPM registry to use for fetching packages. This is not used for publishing releases.
Expand All @@ -63,21 +53,86 @@
* description: E-mail for sending build failure notifications.
*/

def determineReleaseType() {
// This script determines the type of release that
// should be done, given the current date.
// It is possible to specify a different date
// by setting the NOW_DATE environment variable.
// The script will print a text:
// - "minor": if it's determined that a minor release
// should be built.
// - "snapshot.0": if it's determined that a snapshot
// release should be built (specifically, the first
// snapshot of the sprint.
// - "snapshot.1": if it's determined that a snapshot
// release should be built (specifically, the second
// snapshot of the sprint.
// - "snapshot.2": for some sprints with longer duration.
//
// The reference date (base date) can be set in the
// environment variable BASE_DATE. By default, it is the
// last day of Kiali Sprint #14. Starting at end of Sprint #33,
// BASE_DATE is the last day of Sprint #33.
//
// Both NOW_DATE and BASE_DATE should be given in seconds
// since EPOCH. It is assumed that this script is run weekly
// starting in the base date. Running at different timespans
// won't guarantee a good result.
return sh ('''
BASE_DATE=${BASE_DATE:-$(date -d '2018-11-30' '+%s')}
NOW_DATE=${NOW_DATE:-$(date -d 'now' '+%s')}
# At end of Sprint #33, we use it's last day as the base date
cond=$(date -d '2020-01-10' '+%s')
if [ $NOW_DATE -ge $cond ];
then
BASE_DATE=$cond
fi
# Transitional calculations
DATE_DIFF=$(( $NOW_DATE - $BASE_DATE ))
DAYS_ELAPSED=$(( $DATE_DIFF / (24*60*60) ))
WEEKS_ELAPSED=$(( $DAYS_ELAPSED / 7))
# This value will be used to determine the type of the release
WEEKS_MOD3=$(( $WEEKS_ELAPSED % 3 ))
# Sprint #33 is 4 weeks long. Return 'snapshot.2' between Jan 3rd and Jan 9th
if [ $NOW_DATE -ge $(date -d '2020-01-03' '+%s') ] && [ $NOW_DATE -lt $(date -d '2020-01-10' '+%s') ];
then
echo 'snapshot.2'
exit 0
fi
case $WEEKS_MOD3 in
0)
RELEASE_TYPE='minor' ;;
1)
RELEASE_TYPE='snapshot.0' ;;
2)
RELEASE_TYPE='snapshot.1' ;;
esac
# Print the determined type
echo $RELEASE_TYPE
''',
returnStdout: true).trim()
}

node('kiali-build && fedora') {
def backendDir = 'src/github.com/kiali/kiali'
def backendMakefile = 'deploy/jenkins-ci/Makefile'

def uiDir = 'src/github.com/kiali/kiali-ui'
def uiMakefile = 'Makefile.jenkins'

def buildUi = params.SKIP_UI_RELEASE != "y"
def buildBackend = params.SKIP_BACKEND_RELEASE != "y"
def buildServer = params.SKIP_KIALI_SERVER_RELEASE != "y"
def buildOperator = params.SKIP_OPERATOR_RELEASE != "y"
def buildHelm = params.SKIP_HELM_RELEASE != "y" // Temptative value. It's re-assigned later.
def buildSite = params.SKIP_SITE_RELEASE != "y" // Temptative value. It's re-assigned later.
def quayTag = ""

if ( !buildBackend && !buildUi && !buildOperator && !buildHelm && !buildSite ) {
if ( !buildServer && !buildOperator && !buildHelm && !buildSite ) {
currentBuild.result = 'ABORTED'
echo "Nothing to release. Stopping."
return
Expand All @@ -86,7 +141,7 @@ node('kiali-build && fedora') {
try {
cleanWs()
stage('Checkout code') {
if ( buildBackend || buildOperator || buildHelm || buildSite ) {
if ( buildServer ) {
checkout([
$class: 'GitSCM',
branches: [[name: params.RELEASING_BRANCHES]],
Expand All @@ -102,8 +157,7 @@ node('kiali-build && fedora') {

sh "cd ${backendDir}; git config user.email '[email protected]'"
sh "cd ${backendDir}; git config user.name 'kiali-bot'"
}
if ( buildUi ) {

checkout([
$class: 'GitSCM',
branches: [[name: params.RELEASING_BRANCHES]],
Expand All @@ -125,90 +179,68 @@ node('kiali-build && fedora') {
// Determine release type if "auto" was specified
def releaseType = "${params.RELEASE_TYPE}"
if ( releaseType == "auto" ) {
releaseType = sh(
returnStdout: true,
script: "${backendDir}/deploy/jenkins-ci/bin/determine-release-type.sh").trim()
releaseType = determineReleaseType()
}

buildSite = params.SKIP_SITE_RELEASE != "y" && releaseType == "minor"
buildHelm = params.SKIP_HELM_RELEASE != "y" && (releaseType == "minor" || releaseType == "patch")
echo "Resolved release type: ${releaseType}"
echo "Will build back-end? ${buildBackend}"
echo "Will build front-end? ${buildUi}"
echo "Will build back-end? ${buildServer}"
echo "Will build front-end? ${buildServer}"
echo "Will build operator? ${buildOperator}"
echo "Will build Helm charts? ${buildHelm}"
echo "Will build site? ${buildSite}"

withEnv(["PATH+TOOLS=${env.WORKSPACE}/${backendDir}/deploy/jenkins-ci/bin",
"GOPATH=${env.WORKSPACE}",
"[email protected]:${params.BACKEND_REPO}.git",
"[email protected]:kiali-bot/kiali.git",
"BACKEND_PULL_URI=https://api.github.com/repos/${params.BACKEND_REPO}/pulls",
"[email protected]:${params.UI_REPO}.git",
"[email protected]:kiali-bot/kiali-ui.git",
"UI_PULL_URI=https://api.github.com/repos/${params.UI_REPO}/pulls",
"RELEASE_TYPE=${releaseType}"
]) {
parallel backend: {
withEnv(["GOPATH=${env.WORKSPACE}"]) {
stage('Build backend') {
if ( buildBackend ) {
if ( buildServer ) {
withEnv(["PATH+TOOLS=${env.WORKSPACE}/${backendDir}/deploy/jenkins-ci/bin",
"GOPATH=${env.WORKSPACE}",
"[email protected]:${params.BACKEND_REPO}.git",
"[email protected]:kiali-bot/kiali.git",
"BACKEND_PULL_URI=https://api.github.com/repos/${params.BACKEND_REPO}/pulls",
"NPM_DRY_RUN=y", // Just for safety, as we may be running Makefiles for pre-v1.35
"[email protected]:${params.UI_REPO}.git",
"[email protected]:kiali-bot/kiali-ui.git",
"UI_PULL_URI=https://api.github.com/repos/${params.UI_REPO}/pulls",
"RELEASE_TYPE=${releaseType}"
]) {
parallel backend: {
withEnv(["GOPATH=${env.WORKSPACE}"]) {
stage('Build backend') {
sh "make -f ${backendMakefile} -C ${backendDir} backend-build-release"
}
}
stage('Test backend') {
if ( buildBackend ) {
stage('Test backend') {
sh "make -f ${backendMakefile} -C ${backendDir} backend-test"
}
}
}
}, ui: {
stage('Build UI') {
if ( buildUi ) {
}, ui: {
stage('Build UI') {
sh "make -f ${uiMakefile} -C ${uiDir} ui-fix-version ui-build"
}
}
stage('Test UI') {
if ( buildUi ) {
stage('Test UI') {
sh "make -f ${uiMakefile} -C ${uiDir} ui-test"
}
}
},

failFast: true
},
failFast: true

stage('Release kiali-ui to NPM') {
if ( buildUi ) {
withCredentials([string(credentialsId: 'kiali-npm', variable: 'NPM_TOKEN')]) {
sh "make -f ${uiMakefile} -C ${uiDir} ui-npm-publish"
stage('Release Kiali to Container Repositories') {
withCredentials([usernamePassword(credentialsId: 'kiali-quay', passwordVariable: 'QUAY_PASSWORD', usernameVariable: 'QUAY_USER')]) {
sh "make -f ${backendMakefile} -C ${backendDir} backend-push-docker"
quayTag = sh(returnStdout: true, script: "sed -rn 's/^VERSION \\?= v(.*)/v\\1/p' ${backendDir}/Makefile").trim()
}
}
}

stage('Create release cut in front-end repo') {
if ( buildUi ) {
stage('Create release cut in back-end repo') {
withCredentials([string(credentialsId: 'kiali-bot-gh-token', variable: 'GH_TOKEN')]) {
sshagent(['kiali-bot-gh-ssh']) {
sh "make -f ${uiMakefile} -C ${uiDir} ui-push-version-tag ui-prepare-next-version"
sh "make -f ${backendMakefile} -C ${backendDir} backend-push-version-tag backend-prepare-next-version"
}
}
}
}

stage('Release Kiali to Container Repositories') {
if ( buildBackend ) {
withCredentials([usernamePassword(credentialsId: 'kiali-quay', passwordVariable: 'QUAY_PASSWORD', usernameVariable: 'QUAY_USER')]) {
sh "make -f ${backendMakefile} -C ${backendDir} backend-push-docker"
quayTag = sh(returnStdout: true, script: "sed -rn 's/^VERSION \\?= v(.*)/v\\1/p' ${backendDir}/Makefile").trim()
}
}
}

stage('Create release cut in back-end repo') {
if ( buildBackend ) {
stage('Create release cut in front-end repo') {
withCredentials([string(credentialsId: 'kiali-bot-gh-token', variable: 'GH_TOKEN')]) {
sshagent(['kiali-bot-gh-ssh']) {
sh "make -f ${backendMakefile} -C ${backendDir} backend-push-version-tag backend-prepare-next-version"
sh "make -f ${uiMakefile} -C ${uiDir} ui-push-version-tag ui-prepare-next-version"
}
}
}
Expand Down
44 changes: 3 additions & 41 deletions deploy/jenkins-ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,6 @@ more in _[The Pipelines](#the-pipelines)_ section

The `kiali-release` pipeline uses some support files:

* The [bin/determine-release-type.sh](bin/determine-release-type.sh)
is used to automatically resolve what kind of release needs to be
build, assuming the Pipeline runs weekly. Read the script
to learn more about how it works.
* The [bin/jq](https://stedolan.github.io/jq/) v1.6 and
[bin/semver](https://github.com/fsaintjacques/semver-tool) v2.1.0
are tools used to properly set version strings when building releases.
Expand Down Expand Up @@ -160,30 +156,6 @@ Note that the Helm release always builds and pushes to the "master" branch
because that is the branch that GitHub Pages gets the content for the Helm
Chart Repository HTTP server.

### Building a patch release omitting the front-end

First, make sure that all fixes are properly committed to the
back-end repository (don't change version numbers).

In the current Kiali project workflow, patch releases are built off
from a version branch rather than the `master` branch.

Set the Pipeline parameters as follows:

* RELEASE_TYPE: Use `patch`.
* RELEASING_BRANCHES: The branch name of the repositories to
generate the release from; e.g. `refs/heads/v0.20`. The build
assumes that all repositories have a branch with this name.
* SKIP_UI_RELEASE: Set to `y`.

The front-end that will be bundled in the container image will be the version
specified in the main Makefile
(e.g: https://github.com/kiali/kiali/blob/v0.20.0/Makefile#L16). The front-end
will be downloaded from the NPM registry. If you want to bundle a different
front-end version, use the UI_VERSION parameter; for example.

* UI_VERSION: `0.19.0`

### Building a major release

In the current Kiali project workflow, major releases are built off
Expand Down Expand Up @@ -233,8 +205,7 @@ repositories.
There is a set of _SKIP\_*\_RELEASE_ parameters that allow individual control
about what should and should not be built:

* SKIP_BACKEND_RELEASE: Forces to omit the back-end build.
* SKIP_UI_RELEASE: Forces to omit the front-end build.
* SKIP_SERVER_RELEASE: Forces to omit the server build (front-end and back-end).
* SKIP_OPERATOR_RELEASE: Forces to omit the operator build.
* SKIP_HELM_RELEASE: Forces to omit the helm charts build.
* SKIP_SITE_RELEASE: Forces to omit the website build.
Expand Down Expand Up @@ -267,30 +238,22 @@ first check if the `kiali-release` pipeline finished successfully. If it
didn't, pass through the following checks (in order) to know how to proceed
to recover the build. Do the suggested action of the first check that is not OK:

1. Is the front-end release properly
[published in NPM](https://www.npmjs.com/package/@kiali/kiali-ui?activeTab=versions)?
If not:
* Retry the build. No special handling required.
1. Is the front-end version properly tagged in the repository?
(see https://github.com/kiali/kiali-ui/tags). If not:
* Manually create the tag setting the version in package.json
to the published one.
* For builds other than snapshots, manually update the package.json file
of the master/version branch to prepare it for the next release.
* Retry the build but set parameters SKIP_UI_RELEASE to `y` and UI_VERSION
to the version of the front-end that got published in NPM.
1. In the front-end repository, is the package.json correctly updated and
prepared for the next version? If not:
* This check does not apply for snapshot builds.
* Manually update the package.json file of the master/version branch to
prepare it for the next release.
* Retry the build but set parameters SKIP_UI_RELEASE to `y` and UI_VERSION
to the version of the front-end that got published in NPM.
* Retry the build (TODO: how?).
1. Are the container images present in Quay.io?
Is the back-end version properly tagged (see https://github.com/kiali/kiali/tags)?
If not to any of these questions:
* Retry the build but set parameters SKIP_UI_RELEASE to `y` and UI_VERSION
to the version of the front-end that got published in NPM.
* Retry the build (TODO: how?).
1. Is the back-end release properly created in https://github.com/kiali/kiali/releases?
* Don't retry the build.
* Manually [create the version entry in GitHub](https://github.com/kiali/kiali/releases/new).
Expand Down Expand Up @@ -337,7 +300,6 @@ When running the build, set the following parameters:
e.g. `quay.io/edgarhz/kiali`.
* QUAY_OPERATOR_NAME: Use your own Quay.io repository for the operator;
e.g. `quay.io/edgarhz/kiali-operator`.
* NPM_DRY_RUN: Set to `y`.

Once you run the first test build, if you need to run more test builds,
you may want to use the _Rebuild_ option to avoid setting these
Expand Down
Loading

0 comments on commit 5e0b5bb

Please sign in to comment.