Deploy Quay Operator TNG with no default storage class set

[jfroment]

Hello,

I'm trying to deploy Quay Operator (TNG version) on a OCP 4.5.15.
I'm following the guidelines provided in the README (Getting Started section), but it seems that databases PVC are not bound.
On this cluster, there is no default storage class, so that developers are forced to specify one in each of their templates.

But I'm feeling TNG operator does not support this, because:

and so

I set up RHOCP operator, alongside NooBaa datastore object (which is ready). There is no ObjectBucketClaim created, but I can see that there is a new StorageClass provided by NooBaa:

Is there a way to force TNG operator to set this storageclass as the one used to provision PVC? Or can I use my own storage classes for the databases PVC, and only use NooBaa as PV for Quay pod? If so, am I forced to set the databases as "unmanaged"?

Thanks in advance!

[alecmerdler]

Hi @jfroment, thanks for raising this issue! Is there a reason why this cluster doesn't have a default storage class?

[jfroment]

We have different types of storage backends (nfs, block...) and by design, we want to ensure that developers specify (in their templates) the correct storage class according to the environment and type of storage they need for their apps.

With Quay operator v3.3.2 it was not a problem as we could specify the storage class for each component.

I guess I could still label one of the storageclass as "default" in our Quay deployment CI, and then unset it. But that seems to be more of a workaround than a proper configuration.

Will it have an impact on Quay storage as well?
For now quay main deployment replica is set to 0, but I imagine it will be scaled up when other pods (like dbs) are ready?

[alecmerdler]

Thanks for giving more context. I think this is worth investigating.

One of the main guiding principles of this Operator is to be as opinionated as possible as to how Quay is deployed/managed on Kubernetes. This makes it easier to develop, support, and run a cloud native application as complex as Quay. Basically, the less things you can configure, the less things can be misconfigured. Since our team is also responsible for running Quay.io (which runs on Kubernetes), we can take a lot of our learnings of running Quay at a massive scale to craft the ideal deployment configurations for Kubernetes. But feedback like this is important for us to consider as the Quay Operator matures.

When developing features for the Quay Operator, we will try to avoid implementing things that Kubernetes (or other tools) already do well. I noticed this issue for setting the default storage class for a namespace; unfortunately, it doesn't seem to have received much attention. However, it's possible that we could adopt our own solution of annotating the StorageClass which the Operator should use. We try to minimize the API surface of the QuayRegistry object to keep it as simple as possible.

We'll keep this issue open and triage it as part of improvements for our future work. Let us know if you have any ideas that would fit your workflow!

[alecmerdler]

To answer your second question: yes, the Quay Deployment will be scaled to zero as the migration Deployment runs database migrations. It will also be scaled to zero if there are any Quay config issues, which will be surfaced in status.conditions of the QuayRegistry.

[jfroment]

Thanks for this great explanation!
I totally understand the opiniated way of doing things.
I managed to get around this issue by setting the default annotation on a given storage class in my CI deployment scripts, and by unsetting it after deployment is done.
I also had an issue regarding Rolling Update on the DB deployments with this particular storage class which did not support RWX as access mode, but "oc patch" did the trick on these deployments to define a Recreate Strategy.

[MrMEEE]

I second the opinion, that we should not be forced to use the default storageclass..

[MrMEEE]

Is there a workaround for forcing a specific storageclass, or do I have to modify the operator?

[therevoman]

Is there a supported way to specify the storageclass?

[TehreemNisa]

what would be the best to define the StorageClass for Quay managed pvc? as we dont want to use default StorageClass

[lucadentella]

+1 on this: in a cluster with several storage classes, it's IMHO very important a way to specify which storage class(es) the quay components should use.

Thanks

[kbreit-insight]

+1 on this