Can't access tenant console when using encryption

[ClementGautier]

Hi, I'm trying to deploy a Minio instance using Minio Operator using KES encryption and can't get the embeded console.

Expected Behavior

Well, I'd expect the iframes to work out of the box or to give me a hint on what's wrong.

Current Behavior

When I go to the tenant console I get the following:

and in the operator console pod I get the following log

proxy.go:183: couldn't login to tenant and get cookie

Possible Solution

🤷

Steps to Reproduce (for bugs)

Install the chart with those values

secrets:
  existingSecret: minio-keys-storage-g
tenant:
  metrics:
    enabled: true
  name: storage-g
  configuration:
    name: minio-keys-storage-g
  certificate:
    requestAutoCert: false
    externalCertSecret:
      - name: minio-tls
        type: kubernetes.io/tls
    externalClientCertSecrets:
      - name: minio-tls
        type: kubernetes.io/tls
    externalCaCertSecret:
      - name: minio-tls
        type: kubernetes.io/tls
  pools:
  - servers: 4
    name: pool-0
    volumesPerServer: 4
    size: 1000Gi
    storageClassName: silver-xfs
  kes:
    image:
      repository: quay.io/minio/kes
      tag: 2023-10-03T00-48-37Z
      pullPolicy: IfNotPresent
    replicas: 2
    configuration: |-
      version: v1
      address: :7373
      admin:
        identity: _ # Effectively disabled since no root identity necessary.
      tls:
        key: /tmp/kes/server.key   # Path to the TLS private key
        cert: /tmp/kes/server.crt # Path to the TLS certificate
        proxy:
          identities: []
          header:
            cert: X-Tls-Client-Cert
      policy:
        kes:
          allow:
          - /v1/api
          - /v1/key/create/*
          - /v1/key/generate/*
          - /v1/key/decrypt/*
          - /v1/key/bulk/decrypt/*
          identities:
          - ${MINIO_KES_IDENTITY}
      cache:
        expiry:
          any: 5m0s
          unused: 20s
      log:
        error: on
        audit: off
      keystore:
        vault:
          endpoint: "https://vault.vault.svc.cluster.local:8200"
          prefix: "encryption-keys"
          kubernetes:
            engine: ""
            role: kes
            jwt:  /var/run/secrets/kubernetes.io/serviceaccount/token
            retry: 15s
          tls:
            ca: "/tmp/kes/ca.crt"
          status:
            ping: 10s
    keyName: "storage-g"
    imagePullPolicy: "IfNotPresent"
    externalCertSecret: null
    clientCertSecret:
      name: kes-tls
      type: kubernetes.io/tls
    resources: { }
    nodeSelector: { }
    affinity:
      nodeAffinity: { }
      podAffinity: { }
      podAntiAffinity: { }
    tolerations: [ ]
    annotations:
      secret.reloader.stakater.com/reload: "kes-tls,kes-configuration"
    labels: { }
    serviceAccountName: "kes"
    securityContext:
      runAsUser: 1000
      runAsGroup: 1000
      runAsNonRoot: true
      fsGroup: 1000

and here is the certificates I generated using cert-manager:

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: kes-tls
  namespace: minio-tenant
spec:
  secretName: kes-tls
  duration: 2160h #90d
  renewBefore: 720h # 30d
  dnsNames:
  - storage-g-kes-hl-svc.minio-tenant
  - storage-g-kes-hl-svc.minio-tenant.svc
  - storage-g-kes-hl-svc.minio-tenant.svc.cluster.local
  ipAddresses:
  - 127.0.0.1
  issuerRef:
    name: my-root-ca
    kind: ClusterIssuer
  commonName: system:node:kes.minio-tenant.svc.cluster.local
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: minio-tls
  namespace: minio-tenant
spec:
  secretName: minio-tls
  duration: 2160h #90d
  renewBefore: 720h # 30d
  dnsNames:
  - "storage-g-hl.minio-tenant"
  - "storage-g-hl.minio-tenant.svc"
  - "storage-g-hl.minio-tenant.svc.cluster.local"
  - "*.storage-g-hl.minio-tenant.svc.cluster.local"
  - "minio.minio-tenant"
  - "minio.minio-tenant.svc"
  - "minio.minio-tenant.svc.cluster.local"
  - "*.minio-tenant.svc.cluster.local"
  ipAddresses:
  - 127.0.0.1
  issuerRef:
    name: my-root-ca
    kind: ClusterIssuer

I also added the ca.crt of my ca in the operator-ca-tls secret in the operator namespace as told in the tls documentation.

Context

I'm trying to deploy an instance using the Operator deployed using the Helm chart.
Everything works except for the console.

Regression

I'm not sure, I haven't tested previous version to see if it worked

Your Environment

Chart version: 5.0.10 for both tenant and operator
Uses minio version quay.io/minio/minio:RELEASE.2023-10-07T15-07-38Z and kes version quay.io/minio/kes:2023-10-03T00-48-37Z

NB: thanks for your help 🙏

[jiuker]

@ClementGautier Could you replica this with minio RELEASE.2023-01-02T09-40-09Z?

[ClementGautier]

It works !!! Thank you @jiuker, I should've started with that ^^

[harshavardhana]

It works !!! Thank you @jiuker, I should've started with that ^^

Thanks for letting us know @ClementGautier