Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

6.0 helm tenant console service not created #2231

Closed
JamesAtIntegratnIO opened this issue Jul 22, 2024 · 4 comments
Closed

6.0 helm tenant console service not created #2231

JamesAtIntegratnIO opened this issue Jul 22, 2024 · 4 comments
Labels
community triage

Comments

@JamesAtIntegratnIO
Copy link

JamesAtIntegratnIO commented Jul 22, 2024 •
edited
Loading

When creating a new Tenant with Helm Release 6.0.0. The console service is never created in the Tenant.

Expected Behavior

When creating a new tenant I expect a service for {{ .Values.tenant.name }}-console to be created.

Current Behavior

Service is not created.

Listing of created resources

kubectl -n minio-tenant get all
NAME                                                    READY   STATUS      RESTARTS   AGE
pod/default-default-pool-0                              2/2     Running     0          17h
pod/minio-user-creation-job-add-admin-user-59np9        0/1     Completed   0          16h
pod/minio-user-creation-job-admin-policy-create-gbxbg   0/1     Completed   0          16h

NAME                      TYPE           CLUSTER-IP       EXTERNAL-IP                            PORT(S)    AGE
service/default-bucket    ExternalName   <none>           minio.minio-tenant.svc.cluster.local   443/TCP    17h
service/default-hl        ClusterIP      None             <none>                                 9000/TCP   17h
service/minio             ClusterIP      10.108.198.152   <none>                                 443/TCP    17h
service/terraform-state   ExternalName   <none>           minio.minio-tenant.svc.cluster.local   443/TCP    17h

NAME                                    READY   AGE
statefulset.apps/default-default-pool   1/1     17h

values.yaml

secrets:
  name: &name default-tenant-config
tenant:
  name: default
  configuration: 
    name: *name
  pools:
  - servers: 1
    name: default-pool
    volumesPerServer: 1
    volumeClaimTemplate:
      metadata:
        name: data
      spec:
        accessModes:
          - ReadWriteOnce
    storageClassName: nfs-client
    size: 50Gi
    containerSecurityContext:
      runAsUser: 99
      runAsGroup: 100
      runAsNonRoot: true
      allowPrivilegeEscalation: false
      capabilities:
        drop:
          - ALL
      seccompProfile:
        type: RuntimeDefault

  env: 
    - name: MINIO_DOMAIN
      value: "minio.integratn.tech"
    - name: MINIO_BROWSER_REDIRECT_URL
      value: "https://console.integratn.tech"
    - name: MINIO_SERVER_URL
      value: "https://minio.integratn.tech"
  exposeServices:
    console: false
    minio: false
  buckets:
    - name: default-bucket
      region: us-home-1
      objectLock: false
    - name: terraform-state
      region: us-home-1
      objectLock: true
  features:
    bucketDNS: true
  
ingress:
  api:
    enabled: true
    ingressClassName: "nginx"
    labels: { }
    annotations: 
      cert-manager.io/cluster-issuer: letsencrypt-prod
      reloader.stakater.com/auto: "true"
      nginx.ingress.kubernetes.io/proxy-ssl-verify: "off"
      nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
      nginx.ingress.kubernetes.io/rewrite-target: /
      nginx.ingress.kubernetes.io/proxy-body-size: "0"
      nginx.ingress.kubernetes.io/server-snippet: |
        client_max_body_size 0;
      nginx.ingress.kubernetes.io/configuration-snippet: |
        chunked_transfer_encoding off;
    tls: 
      - hosts:
          - &host minio.integratn.tech
          - "*.minio.integratn.tech"
        secretName: minio-tls
    host: *host
    path: /
    pathType: Prefix
  console:
    enabled: true
    ingressClassName: "nginx"
    labels: { }
    annotations: 
      cert-manager.io/cluster-issuer: letsencrypt-prod
      reloader.stakater.com/auto: "true"
      nginx.ingress.kubernetes.io/proxy-ssl-verify: "off"
      nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
      nginx.ingress.kubernetes.io/rewrite-target: /
      nginx.ingress.kubernetes.io/proxy-body-size: "0"
      nginx.ingress.kubernetes.io/server-snippet: |
    tls: 
      - hosts:
          - &host console.integratn.tech

        secretName: minio-prod-tls
    host: *host
    path: /
    pathType: Prefix

Argo Application Manifest

project: default
destination:
  namespace: minio-tenant
  name: monitoring-cluster
syncPolicy:
  automated: {}
  syncOptions:
    - CreateNamespace=true
    - ServerSideApply=true
sources:
  - repoURL: 'https://operator.min.io'
    targetRevision: 6.0.0
    helm:
      valueFiles:
        - >-
          $values/gitops/workloads/monitoring-cluster/minio-tenant/manifests/values.yaml
      releaseName: minio-tenant
      ignoreMissingValueFiles: true
    chart: tenant
  - repoURL: '[email protected]:jamesAtIntegratnIO/gitops-homelab.git'
    path: gitops/workloads/monitoring-cluster/minio-tenant/manifests
    targetRevision: main
    ref: values

Startup Logs

I0721 23:05:25.760004       1 controller.go:83] Starting MinIO Operator
I0721 23:05:25.762010       1 main-controller.go:311] Setting up event handlers
I0721 23:05:25.790382       1 main-controller.go:541] Using Kubernetes CSR Version: v1
I0721 23:05:25.790442       1 main-controller.go:561] Waiting for STS API to start
I0721 23:05:25.790463       1 leaderelection.go:250] attempting to acquire leader lease minio-operator/minio-operator-lock...
I0721 23:05:25.790542       1 main-controller.go:417] Starting STS API server
I0721 23:05:25.796582       1 tls.go:54] Waiting for the sts certificates secret to be issued
I0721 23:05:25.808780       1 leaderelection.go:260] successfully acquired lease minio-operator/minio-operator-lock
I0721 23:05:25.808877       1 main-controller.go:592] minio-operator-798bc4b9c-swmcj: I am the leader, applying leader labels on myself
I0721 23:05:25.809042       1 main-controller.go:446] Waiting for Upgrade Server to start
I0721 23:05:25.809208       1 main-controller.go:406] Starting HTTP Upgrade Tenant Image server
I0721 23:05:25.810289       1 main-controller.go:450] Starting Tenant controller
I0721 23:05:25.810383       1 main-controller.go:453] Waiting for informer caches to sync
I0721 23:05:25.810478       1 main-controller.go:464] Starting workers and Job workers
I0721 23:05:25.810743       1 main-controller.go:483] STS Autocert is enabled, starting API certificate setup.
I0721 23:05:25.822120       1 tls.go:127] sts-tls TLS secret not found: secrets "sts-tls" not found
I0721 23:05:25.853397       1 csr.go:183] Start polling for certificate of csr/sts-minio-operator-csr, every 5s, timeout after 20m0s
I0721 23:05:30.852886       1 minio.go:185] Generating private key
I0721 23:05:30.853039       1 minio.go:198] Generating CSR with CN=*.default-hl.minio-tenant.svc.cluster.local
I0721 23:05:30.858637       1 csr.go:209] Certificate successfully fetched, creating secret with Private key and Certificate
I0721 23:05:30.872363       1 tls.go:130] Waiting for the sts certificates to be issued waiting for sts cert
I0721 23:05:30.873495       1 csr.go:183] Start polling for certificate of csr/default-minio-tenant-csr, every 5s, timeout after 20m0s
I0721 23:05:30.873888       1 event.go:377] Event(v1.ObjectReference{Kind:"Tenant", Namespace:"minio-tenant", Name:"default", UID:"cc9dedef-be82-4443-8586-e0e0956d3b12", APIVersion:"minio.min.io/v2", ResourceVersion:"6074", FieldPath:""}): type: 'Normal' reason: 'CSRCreated' MinIO CSR Created
I0721 23:05:35.879068       1 csr.go:209] Certificate successfully fetched, creating secret with Private key and Certificate
I0721 23:05:40.891702       1 event.go:377] Event(v1.ObjectReference{Kind:"Tenant", Namespace:"minio-tenant", Name:"default", UID:"cc9dedef-be82-4443-8586-e0e0956d3b12", APIVersion:"minio.min.io/v2", ResourceVersion:"6132", FieldPath:""}): type: 'Normal' reason: 'SvcCreated' MinIO Service Created
I0721 23:05:40.904870       1 status.go:55] Hit conflict issue, getting latest version of tenant
I0721 23:05:40.939693       1 event.go:377] Event(v1.ObjectReference{Kind:"Tenant", Namespace:"minio-tenant", Name:"default", UID:"cc9dedef-be82-4443-8586-e0e0956d3b12", APIVersion:"minio.min.io/v2", ResourceVersion:"6138", FieldPath:""}): type: 'Normal' reason: 'SvcCreated' Headless Service created
I0721 23:05:40.967822       1 sync.go:151] created key minio-tenant/default-sa kind /v1, Kind=ServiceAccount diff [ObjectMeta.UID:  != effcfa0b-ec44-49e1-812a-ef91db428a3f ObjectMeta.ResourceVersion:  != 6143 ObjectMeta.CreationTimestamp.Time: 0001-01-01 00:00:00 +0000 UTC != 2024-07-21 23:05:40 +0000 UTC ObjectMeta.OwnerReferences: <nil slice> != [{minio.min.io/v2 Tenant default cc9dedef-be82-4443-8586-e0e0956d3b12 0xc000de714d 0xc000de714e}] ObjectMeta.ManagedFields: <nil slice> != [{minio-operator Update v1 2024-07-21 23:05:40 +0000 UTC FieldsV1 {"f:metadata":{"f:ownerReferences":{".":{},"k:{\"uid\":\"cc9dedef-be82-4443-8586-e0e0956d3b12\"}":{}}}} }]]
I0721 23:05:41.004606       1 sync.go:151] created key minio-tenant/default-role kind rbac.authorization.k8s.io/v1, Kind=Role diff [ObjectMeta.UID:  != d33a554c-eafa-45fb-a57e-5d611c2b9299 ObjectMeta.ResourceVersion:  != 6145 ObjectMeta.CreationTimestamp.Time: 0001-01-01 00:00:00 +0000 UTC != 2024-07-21 23:05:40 +0000 UTC ObjectMeta.OwnerReferences: <nil slice> != [{minio.min.io/v2 Tenant default cc9dedef-be82-4443-8586-e0e0956d3b12 0xc000de75cd 0xc000de75ce}] ObjectMeta.ManagedFields: <nil slice> != [{minio-operator Update rbac.authorization.k8s.io/v1 2024-07-21 23:05:40 +0000 UTC FieldsV1 {"f:metadata":{"f:ownerReferences":{".":{},"k:{\"uid\":\"cc9dedef-be82-4443-8586-e0e0956d3b12\"}":{}}},"f:rules":{}} }]]
I0721 23:05:41.054480       1 sync.go:151] created key minio-tenant/default-binding kind rbac.authorization.k8s.io/v1, Kind=RoleBinding diff [ObjectMeta.UID:  != 6e4872c7-41d5-4033-afe6-323333c72d0e ObjectMeta.ResourceVersion:  != 6148 ObjectMeta.CreationTimestamp.Time: 0001-01-01 00:00:00 +0000 UTC != 2024-07-21 23:05:41 +0000 UTC ObjectMeta.OwnerReferences: <nil slice> != [{minio.min.io/v2 Tenant default cc9dedef-be82-4443-8586-e0e0956d3b12 0xc000de7c5d 0xc000de7c5e}] ObjectMeta.ManagedFields: <nil slice> != [{minio-operator Update rbac.authorization.k8s.io/v1 2024-07-21 23:05:41 +0000 UTC FieldsV1 {"f:metadata":{"f:ownerReferences":{".":{},"k:{\"uid\":\"cc9dedef-be82-4443-8586-e0e0956d3b12\"}":{}}},"f:roleRef":{},"f:subjects":{}} }]]
I0721 23:05:41.086863       1 status.go:89] Hit conflict issue, getting latest version of tenant
I0721 23:05:41.104411       1 main-controller.go:978] Detected we are updating a legacy tenant deployment
I0721 23:05:41.120178       1 main-controller.go:1023] 'minio-tenant/default': Deploying pool default-pool
I0721 23:05:41.246574       1 event.go:377] Event(v1.ObjectReference{Kind:"Tenant", Namespace:"minio-tenant", Name:"default", UID:"cc9dedef-be82-4443-8586-e0e0956d3b12", APIVersion:"minio.min.io/v2", ResourceVersion:"6151", FieldPath:""}): type: 'Normal' reason: 'PoolCreated' Tenant pool default-pool created
I0721 23:05:42.428608       1 event.go:377] Event(v1.ObjectReference{Kind:"Tenant", Namespace:"minio-tenant", Name:"default", UID:"cc9dedef-be82-4443-8586-e0e0956d3b12", APIVersion:"minio.min.io/v2", ResourceVersion:"6175", FieldPath:""}): type: 'Warning' reason: 'WaitingMinIOIsHealthy' Waiting for MinIO to be ready: %!s(<nil>)
I0721 23:05:42.630156       1 monitoring.go:123] 'minio-tenant/default' Failed to get cluster health: Get "https://minio.minio-tenant.svc.cluster.local/minio/health/cluster": dial tcp 10.108.198.152:443: connect: connection refused
I0721 23:05:43.600410       1 monitoring.go:123] 'minio-tenant/default' Failed to get cluster health: Get "https://minio.minio-tenant.svc.cluster.local/minio/health/cluster": dial tcp 10.108.198.152:443: connect: connection refused
I0721 23:05:44.616019       1 monitoring.go:123] 'minio-tenant/default' Failed to get cluster health: Get "https://minio.minio-tenant.svc.cluster.local/minio/health/cluster": dial tcp 10.108.198.152:443: connect: connection refused
I0721 23:05:45.631310       1 monitoring.go:123] 'minio-tenant/default' Failed to get cluster health: Get "https://minio.minio-tenant.svc.cluster.local/minio/health/cluster": dial tcp 10.108.198.152:443: connect: connection refused
I0721 23:05:45.873975       1 minio-services.go:156] Headless Services don't match: service ports don't match
I0721 23:05:45.878890       1 event.go:377] Event(v1.ObjectReference{Kind:"Tenant", Namespace:"minio-tenant", Name:"default", UID:"cc9dedef-be82-4443-8586-e0e0956d3b12", APIVersion:"minio.min.io/v2", ResourceVersion:"6199", FieldPath:""}): type: 'Normal' reason: 'Updated' Headless Service Updated
I0721 23:05:45.917387       1 event.go:377] Event(v1.ObjectReference{Kind:"Tenant", Namespace:"minio-tenant", Name:"default", UID:"cc9dedef-be82-4443-8586-e0e0956d3b12", APIVersion:"minio.min.io/v2", ResourceVersion:"6199", FieldPath:""}): type: 'Warning' reason: 'WaitingMinIOIsHealthy' Waiting for MinIO to be ready: %!s(<nil>)
I0721 23:05:46.644956       1 monitoring.go:123] 'minio-tenant/default' Failed to get cluster health: Get "https://minio.minio-tenant.svc.cluster.local/minio/health/cluster": dial tcp 10.108.198.152:443: connect: connection refused
I0721 23:05:47.663346       1 monitoring.go:123] 'minio-tenant/default' Failed to get cluster health: Get "https://minio.minio-tenant.svc.cluster.local/minio/health/cluster": dial tcp 10.108.198.152:443: connect: connection refused
I0721 23:05:48.677405       1 monitoring.go:123] 'minio-tenant/default' Failed to get cluster health: Get "https://minio.minio-tenant.svc.cluster.local/minio/health/cluster": dial tcp 10.108.198.152:443: connect: connection refused
I0721 23:05:49.689328       1 monitoring.go:123] 'minio-tenant/default' Failed to get cluster health: Get "https://minio.minio-tenant.svc.cluster.local/minio/health/cluster": dial tcp 10.108.198.152:443: connect: connection refused
I0721 23:05:50.702849       1 monitoring.go:123] 'minio-tenant/default' Failed to get cluster health: Get "https://minio.minio-tenant.svc.cluster.local/minio/health/cluster": dial tcp 10.108.198.152:443: connect: connection refused
I0721 23:05:51.002259       1 event.go:377] Event(v1.ObjectReference{Kind:"Tenant", Namespace:"minio-tenant", Name:"default", UID:"cc9dedef-be82-4443-8586-e0e0956d3b12", APIVersion:"minio.min.io/v2", ResourceVersion:"6228", FieldPath:""}): type: 'Warning' reason: 'WaitingMinIOIsHealthy' Waiting for MinIO to be ready: %!s(<nil>)
I0721 23:09:07.144181       1 helper.go:760] Successfully created bucket default-bucket
I0721 23:09:07.179773       1 helper.go:760] Successfully created bucket terraform-state
I0721 23:09:07.195007       1 event.go:377] Event(v1.ObjectReference{Kind:"Tenant", Namespace:"minio-tenant", Name:"default", UID:"cc9dedef-be82-4443-8586-e0e0956d3b12", APIVersion:"minio.min.io/v2", ResourceVersion:"7105", FieldPath:""}): type: 'Normal' reason: 'BucketsCreated' Buckets created
I0721 23:09:07.202816       1 status.go:55] Hit conflict issue, getting latest version of tenant
I0721 23:09:07.226622       1 pdb.go:191] PodDisruptionBudget: v1

Possible Solution

Steps to Reproduce (for bugs)

My operator deployment

My tenant deployment

Context

Regression

Your Environment

  • Version used (minio-operator): Helm 6.0
  • Environment name and version (e.g. kubernetes v1.17.2): Talos Kubernetes 1.30.0
  • Server type and version: Proxmox 8 VM
  • Operating System and version (uname -a): Talos 1.7.5
  • Link to your deployment file:
@cesnietor
Copy link
Contributor

this should be fixed here: #2230
we'll make a release soon.

@bkinzle
Copy link

bkinzle commented Jul 25, 2024 •
edited
Loading

So just to be clear, will these instructions continue to work after the fix?

This is for the Operator console, before a Tenant is even created where it assumes a service account + secret has been created for the Operator console... or is the Operator console what was intended to be removed so these instructions are no longer valid?

@JamesAtIntegratnIO
Copy link
Author

I don't think those instructions are any good. There is no long a console in the operator. So step 5 on is no longer usable.

@ramondeklein
Copy link
Contributor

ramondeklein commented Jul 25, 2024 •
edited
Loading

@bkinzle @JamesAtIntegratnIO Documentation for operator will be updated (our docs team is working on it). There is no operator console in v6 anymore, so all instructions that will describe the operator console will be removed as well. Creating tenants should now be done directly by creating Tenant custom resources or using the tenant helm chart.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
community triage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ramondeklein @cesnietor @JamesAtIntegratnIO @bkinzle