Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] kubeblocks helm upgrade-hook-job failed #8761

Open
juchaosong opened this issue Jan 8, 2025 · 1 comment · May be fixed by #8780
Open

[BUG] kubeblocks helm upgrade-hook-job failed #8761

juchaosong opened this issue Jan 8, 2025 · 1 comment · May be fixed by #8780
Assignees
Labels
kind/bug Something isn't working
Milestone

Comments

@juchaosong
Copy link
Contributor

Describe the bug
helm upgrade failed.

To Reproduce
After install kubeblocks using helm with version 0.9.2, the upgrade-hook-job will stuck with pre-upgrade-job container panic.

Additional context
The helm logs:

middleware, kubeblock-upgrade-hook-job, Job (batch) has been added:
- 
+ # Source: kubeblocks/templates/kubeblocks-upgrade-hook.yaml
+ apiVersion: batch/v1
+ kind: Job
+ metadata:
+   name: kubeblock-upgrade-hook-job
+   labels:
+     helm.sh/chart: kubeblocks-0.9.2
+     app.kubernetes.io/name: kubeblocks
+     app.kubernetes.io/instance: kubeblock
+     app.kubernetes.io/version: "0.9.2"
+     app.kubernetes.io/managed-by: Helm
+   annotations:
+     "helm.sh/hook": pre-upgrade
+     "helm.sh/hook-delete-policy": "before-hook-creation,hook-succeeded"
+ spec:
+   ttlSecondsAfterFinished: 3600
+   template:
+     metadata:
+       name: kubeblock-upgrade
+       labels:
+         helm.sh/chart: kubeblocks-0.9.2
+         app.kubernetes.io/name: kubeblocks
+         app.kubernetes.io/instance: kubeblock
+         app.kubernetes.io/version: "0.9.2"
+         app.kubernetes.io/managed-by: Helm
+     spec:
+       serviceAccountName: kubeblock-kubeblocks
+       securityContext:
+         runAsNonRoot: true
+       restartPolicy: OnFailure
+       containers:
+         - name: pre-upgrade-job
+           image: "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.2"
+           imagePullPolicy: IfNotPresent
+           command:
+             - /bin/helm_hook
+           args:
+             - --version=0.9.2
+             - --namespace=middleware
+         - name: delete-ops
+           image: "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:0.9.2"
+           imagePullPolicy: IfNotPresent
+           command:
+             - sh
+             - -c
+             - |
+               opsDefs=$(kubectl get opsdefinitions.apps.kubeblocks.io | awk 'NR>1 {print $1}')
+               for opsDef in ${opsDefs}; do
+                 if [ $(kubectl get opsdefinitions.apps.kubeblocks.io ${opsDef} -oyaml | grep "  jobSpec:") ]; then
+                   kubectl patch opsdefinitions.apps.kubeblocks.io ${opsDef} -p '{"metadata":{"finalizers":null}}' --type=merge
+                   kubectl delete opsdefinitions.apps.kubeblocks.io ${opsDef}
+                 elif [ ! $(kubectl get opsdefinitions.apps.kubeblocks.io ${opsDef} -oyaml | grep "  actions:") ]; then
+                   kubectl patch opsdefinitions.apps.kubeblocks.io ${opsDef} --type=merge -p '{"spec":{"actions":[{"name":"mock","exec":{"podInfoExtractorName":"test","command":["echo 1"]}}]}}'
+                 fi
+               done
+       affinity:
+         nodeAffinity:
+           preferredDuringSchedulingIgnoredDuringExecution:
+           - preference:
+               matchExpressions:
+               - key: kb-controller
+                 operator: In
+                 values:
+                 - "true"
+             weight: 100
+       tolerations:
+         - effect: NoSchedule
+           key: kb-controller
+           operator: Equal
+           value: "true"

The pod logs:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0xb0 pc=0x16fe31c]

goroutine 1 [running]:
github.com/apecloud/kubeblocks/cmd/helmhook/hook.getVersionInfo({0x1dddf80?, 0xc00012c900?}, 0x18?, {0x7fffcf0b993a?, 0xc000497db8?})
 /src/cmd/helmhook/hook/prepare.go:41 +0x3c
github.com/apecloud/kubeblocks/cmd/helmhook/hook.PrepareFor(0xc00012c900)
 /src/cmd/helmhook/hook/prepare.go:29 +0x32
github.com/apecloud/kubeblocks/cmd/helmhook/hook.Workflow.Do(...)
 /src/cmd/helmhook/hook/types.go:100
main.main()
 /src/cmd/helmhook/main.go:80 +0x630
@juchaosong juchaosong added the kind/bug Something isn't working label Jan 8, 2025
@shanshanying
Copy link
Contributor

@sophon-zt PTAL

@shanshanying shanshanying added this to the Release 0.9.2 milestone Jan 8, 2025
@sophon-zt sophon-zt linked a pull request Jan 9, 2025 that will close this issue
@github-actions github-actions bot modified the milestones: Release 0.9.2, Release 0.9.3 Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
3 participants