cdi-operator crashloopbackoff

[AOrps]

What happened:
Failed to pull cdi-operator container image.

What you expected to happen:
I would have expected that after creating the cdi-operator, it would have pulled the image from quay and I would have the cdi-operator running.

How to reproduce it (as minimally and precisely as possible):
I ran this, and no dice:

export VERSION=$(curl -s https://api.github.com/repos/kubevirt/containerized-data-importer/releases/latest | grep '"tag_name":' | sed -E 's/.*"([^"]+)".*/\1/')
wget ttps://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-operator.yaml
kubectl create -f cdi-operator.yaml
wget https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-cr.yaml
kubectl create -f cdi-cr.yaml

Additional context:
It appears that the Image is not sync'd up with the Image ID (as in the Image ID is empty). However the image does exist in quay: https://quay.io/repository/kubevirt/cdi-operator?tab=tags&tag=v1.60.4

After running kubectl describe po cdi-operator-8f5f4b54b-bcnh4:

Name:             cdi-operator-8f5f4b54b-bcnh4
Namespace:        cdi
Priority:         0
Service Account:  cdi-operator
Node:             kubevirt-worker/172.19.0.2
Start Time:       Sat, 18 Jan 2025 20:55:30 -0500
Labels:           cdi.kubevirt.io=cdi-operator
                  name=cdi-operator
                  operator.cdi.kubevirt.io=
                  pod-template-hash=8f5f4b54b
                  prometheus.cdi.kubevirt.io=true
Annotations:      <none>
Status:           Pending
IP:               10.244.1.11
IPs:
  IP:           10.244.1.11
Controlled By:  ReplicaSet/cdi-operator-8f5f4b54b
Containers:
  cdi-operator:
    Container ID:    
    Image:           quay.io/kubevirt/cdi-operator:v1.60.4
    Image ID:        
    Port:            8080/TCP
    Host Port:       0/TCP
    SeccompProfile:  RuntimeDefault
    State:           Waiting
      Reason:        ImagePullBackOff
    Ready:           False
    Restart Count:   0
    Requests:
      cpu:     100m
      memory:  150Mi
    Environment:
      DEPLOY_CLUSTER_RESOURCES:  true
      OPERATOR_VERSION:          v1.60.4
      CONTROLLER_IMAGE:          quay.io/kubevirt/cdi-controller:v1.60.4
      IMPORTER_IMAGE:            quay.io/kubevirt/cdi-importer:v1.60.4
      CLONER_IMAGE:              quay.io/kubevirt/cdi-cloner:v1.60.4
      OVIRT_POPULATOR_IMAGE:     quay.io/kubevirt/cdi-importer:v1.60.4
      APISERVER_IMAGE:           quay.io/kubevirt/cdi-apiserver:v1.60.4
      UPLOAD_SERVER_IMAGE:       quay.io/kubevirt/cdi-uploadserver:v1.60.4
      UPLOAD_PROXY_IMAGE:        quay.io/kubevirt/cdi-uploadproxy:v1.60.4
      VERBOSITY:                 1
      PULL_POLICY:               IfNotPresent
      MONITORING_NAMESPACE:      
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lrrz4 (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       False 
  ContainersReady             False 
  PodScheduled                True 
Volumes:
  kube-api-access-lrrz4:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 CriticalAddonsOnly op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason   Age                   From     Message
  ----     ------   ----                  ----     -------
  Warning  Failed   36m (x47 over 13h)    kubelet  (combined from similar events): Failed to pull image "quay.io/kubevirt/cdi-operator:v1.60.4": rpc error: code = DeadlineExceeded desc = failed to pull and unpack image "quay.io/kubevirt/cdi-operator:v1.60.4": failed to resolve reference "quay.io/kubevirt/cdi-operator:v1.60.4": failed to do request: Head "https://quay.io/v2/kubevirt/cdi-operator/manifests/v1.60.4": dial tcp 54.210.103.166:443: i/o timeout
  Normal   BackOff  93s (x3544 over 14h)  kubelet  Back-off pulling image "quay.io/kubevirt/cdi-operator:v1.60.4"

Environment:

• CDI version (use kubectl get deployments cdi-deployment -o yaml):
  I used kubectl get deployments cdi-operator -o yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2025-01-19T01:55:30Z"
  generation: 1
  labels:
    cdi.kubevirt.io: cdi-operator
    name: cdi-operator
    operator.cdi.kubevirt.io: ""
    prometheus.cdi.kubevirt.io: "true"
  name: cdi-operator
  namespace: cdi
  resourceVersion: "75335"
  uid: c0176d21-19a1-46e3-8ce2-fd3c94934ebf
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      name: cdi-operator
      operator.cdi.kubevirt.io: ""
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        cdi.kubevirt.io: cdi-operator
        name: cdi-operator
        operator.cdi.kubevirt.io: ""
        prometheus.cdi.kubevirt.io: "true"
    spec:
      affinity:
        podAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: cdi.kubevirt.io
                  operator: In
                  values:
                  - cdi-operator
              topologyKey: kubernetes.io/hostname
            weight: 1
      containers:
      - env:
        - name: DEPLOY_CLUSTER_RESOURCES
          value: "true"
        - name: OPERATOR_VERSION
          value: v1.60.4
        - name: CONTROLLER_IMAGE
          value: quay.io/kubevirt/cdi-controller:v1.60.4
        - name: IMPORTER_IMAGE
          value: quay.io/kubevirt/cdi-importer:v1.60.4
        - name: CLONER_IMAGE
          value: quay.io/kubevirt/cdi-cloner:v1.60.4
        - name: OVIRT_POPULATOR_IMAGE
          value: quay.io/kubevirt/cdi-importer:v1.60.4
        - name: APISERVER_IMAGE
          value: quay.io/kubevirt/cdi-apiserver:v1.60.4
        - name: UPLOAD_SERVER_IMAGE
          value: quay.io/kubevirt/cdi-uploadserver:v1.60.4
        - name: UPLOAD_PROXY_IMAGE
          value: quay.io/kubevirt/cdi-uploadproxy:v1.60.4
        - name: VERBOSITY
          value: "1"
        - name: PULL_POLICY
          value: IfNotPresent
        - name: MONITORING_NAMESPACE
        image: quay.io/kubevirt/cdi-operator:v1.60.4
        imagePullPolicy: IfNotPresent
        name: cdi-operator
        ports:
        - containerPort: 8080
          name: metrics
          protocol: TCP
        resources:
          requests:
            cpu: 100m
            memory: 150Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          runAsNonRoot: true
          seccompProfile:
            type: RuntimeDefault
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        runAsNonRoot: true
      serviceAccount: cdi-operator
      serviceAccountName: cdi-operator
      terminationGracePeriodSeconds: 30
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
status:
  conditions:
  - lastTransitionTime: "2025-01-19T01:55:30Z"
    lastUpdateTime: "2025-01-19T01:55:30Z"
    message: Deployment does not have minimum availability.
    reason: MinimumReplicasUnavailable
    status: "False"
    type: Available
  - lastTransitionTime: "2025-01-19T02:05:31Z"
    lastUpdateTime: "2025-01-19T02:05:31Z"
    message: ReplicaSet "cdi-operator-8f5f4b54b" has timed out progressing.
    reason: ProgressDeadlineExceeded
    status: "False"
    type: Progressing
  observedGeneration: 1
  replicas: 1
  unavailableReplicas: 1
  updatedReplicas: 1

• Kubernetes version (use kubectl version):

Client Version: v1.29.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.1

• DV specification: N/A
• Cloud provider or hardware configuration: N/A
• OS (e.g. from /etc/os-release):

PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

• Kernel (e.g. uname -a):

Linux <hostname> 6.5.0-1014-oem #15-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 19 11:03:48 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

• Install tools: N/A
• Others: N/A

[AOrps]

Happy to help with a fix, if you point me in the direction of past art and/or where in the codebase I might need to look at to revamp this. I tried on 2 versions: v1.61.0 & v1.60.4. Also happy to provide more context if needed. Thanks!