support minio as object storage

[abdennour]

Quay requires object storage.
The documentation mentions:

• aws s3
• openstack swift
• Or Ceph Rados ( l love OSDs btw)

Now the question, can you support customizing the endpoint of s3 api because i want to use minio.
While minio is 100% API compatible with AWS S3, i need to provide the endpoint (url) of the minio server

old similar issue : https://access.redhat.com/solutions/3782641

[alecmerdler]

Yes, MinIO will work with Quay using the RadosGWStorage driver option in config.yaml and providing the appropriate values. Our team has used it for development and testing purposes. It is not supported as part of the Red Hat Quay product, however.

Here is a partial config.yaml for using MinIO:

DISTRIBUTED_STORAGE_CONFIG:
  default:
  - RadosGWStorage
  - access_key: <hidden>
    secret_key: <hidden>
    hostname: <hidden>
    bucket_name: <hidden>
    port: 443
    is_secure: true
    storage_path: /datastorage/registry

[abdennour]

@alecmerdler thank you so much! really appreciated!
Could you please do me a favor by sharing the file cconfiguration: not only config.yaml but also the kubernetes manifests (QuayRegistry/QuayEcosystem/ConfigMap/Secret/....) ?
i just need the shape where you can hide irrelevant values (sensitive,.. so on)

[alecmerdler]

Sure, configuring a QuayRegistry to use unmanaged object storage should be very easy. You can either use the config tool UI (found using status.configEditorEndpoint) or by providing spec.configBundleSecret which contains the required Quay config values found above. I won't go into deploying MinIO itself, but configuring Quay to use an existing MinIO deployment basically follows the "unmanaged component" pattern, which looks a bit like this:

whatever.quayregistryyaml

apiVersion: quay.redhat.com/v1
kind: QuayRegistry
metadata: 
  name: whatever
spec:
  configBundleSecret: quay-config-abc
  components:
    - kind: objectstorage
      managed: false

quay-config-abc.secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: quay-config-abc
stringData:
  config.yaml:
    ... <other key/values>
    DISTRIBUTED_STORAGE_CONFIG:
      default:
      - RadosGWStorage
      - access_key: <hidden>
        secret_key: <hidden>
        hostname: <hidden>
        bucket_name: <hidden>
        port: 443
        is_secure: true
        storage_path: /datastorage/registry

[venezia]

FWIW, I use minio using this sort of configuration, but I had an issue.

When I had minio endpoint set up as minio.quay.mydomain.com it would work fine with docker pull, but fail with crictl pull ... but if I set the minio endpoint (nothing else different) as minio-quay.mydomain.com then it would work with crictl pull

The error from crictl pull was

FATA[0000] pulling image: rpc error: code = Unknown desc = Error parsing image configuration: Error fetching blob: invalid status code from registry 400 (Bad Request)

It made me feel like there was a CORS header issue, possibly related to this comment in the code

In other words, a real minio client implementation may be needed ... also perhaps this comment should be in the quay repo and not the operator repo, unsure!

[alecmerdler]

@venezia Indeed, looks like Quay needs to use a real MinIO client for crictl. For Quay we have focused on maintaing compatibility with docker, podman, and rkt (back in the CoreOS days). If this is important for your use case, I recommend either sending an email to [email protected] or filing an issue using our issue tracker.

[abdennour]

i wanna specify also:

• Certificate authority to connect securely with the s3 endponit
• as well as, ability to skip the s3 v4 signature

what are the suitable config keys for that

[vChrisR]

I'm also looking to specify a CA to connect to S3 endpoint. Is it possible to set it?