Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Improve the airgap env preparation script [RHIDP-1442][RHIDP-4415] #705

Open
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

rm3l
Copy link
Member

@rm3l rm3l commented Jan 27, 2025

Description

This improves the airgap env preparation script so that:

  1. it can work on both OCP (regular, ROSA or with hosted control planes) and vanilla K8s clusters.
  2. it can work with partially or fully disconnected environments. For example, to support the fully disconnected environment case, it can mirror everything to disk and start from an existing input folder. This makes it work with bastion hosts.
    It also has the option to leverage the oc-mirror tool since this is the recommended way to mirror on OCP.

Which issue(s) does this PR fix or relate to

PR acceptance criteria

  • Tests
  • Documentation

How to test changes / Special notes to the reviewer

Follow the procedure in https://github.com/rm3l/redhat-developer-hub-operator/blob/RHIDP-4415--airgap-install-script-improvements/.rhdh/docs/airgap.adoc

Note that this introduces new options to the script, but for backward compatibility, the previous ones are preserved (but deprecated) as much as possible (except --helper_mirror_registry_storage, --use_existing_mirror_registry, --prod_operator_package_name, and prod_operator_bundle_name, which no longer make sense here).

Usage
$ .rhdh/scripts/prepare-restricted-environment.sh --help

This script streamlines the installation of the Red Hat Developer Hub Operator in a disconnected OpenShift or Kubernetes cluster.
It supports partially disconnected as well as fully disconnected environments.
In a partially disconnected environment, the host from which this script is executed has access to the Internet and the Red Hat ecosystem catalog,
and can push the images directly to the mirror registry and the cluster.
In a fully disconnected environment however, everything needs to be mirrored to disk first, then transferred to the
disconnected environment (usually via a bastion host), from where we can connect to the mirror registry and the cluster.

Usage:
  .rhdh/scripts/prepare-restricted-environment.sh [OPTIONS]

Options:
  --index-image <operator-index-image>   : Operator index image (default: registry.redhat.io/redhat/redhat-operator-index:v4.17)
  --filter-versions <list>               : Comma-separated list of operator minor versions to keep in the catalog (default: 1.3,1.4)
  --to-registry <registry_url>           : Mirror the images into the specified registry, assuming you are already logged into it.
                                            If this is not set and --to-dir is not set, it will attempt to use the builtin OCP registry
                                            if the target cluster is OCP. Otherwise, it will error out.
                                            It also assumes you are logged into the target cluster as well.
  --to-dir </absolute/path/to/dir>       : Mirror images into the specified directory. Needs to be an absolute path.
                                            This is useful if you are working in a fully disconnected environment and
                                            you must manually transfer the images to your network.
                                            From there, you will be able to re-run this script with '--from-dir' to push
                                            the images to your private registry.
  --from-dir </absolute/path/to/dir>     : Load images from the specified directory. Needs to be an absolute path.
                                            This is useful if you are working in a fully disconnected environment.
                                            In this case, you would use '--to-dir' first to mirror images to a specified directory,
                                            then transfer this dir over to your disconnected network.
                                            From there, you will be able to re-run this script with '--from-dir' to push
                                            the images to your private registry.
  --install-operator <true|false>        : Install the RHDH operator right after creating the CatalogSource (default: true)
  --extra-images <list>                  : Comma-separated list of extra images to mirror
  --use-oc-mirror <true|false>           : Whether to use the 'oc-mirror' tool (default: false).
                                            This is the recommended way for mirroring on regular OpenShift clusters.
                                            Bear in mind however that this relies on resources like ImageContentSourcePolicy,
                                            which don't seem to work well on ROSA clusters or clusters with hosted control
                                            planes (like HyperShift or Red Hat OpenShift on IBM Cloud).

Examples:

  # Install the Catalog Source by pushing the images to the internal OCP mirror registry,
  #   because it detected that it is connected to an OCP cluster.
  .rhdh/scripts/prepare-restricted-environment.sh

  # Install the Catalog Source by pushing the images to the specified mirror registry, assuming the user is already logged into it.
  .rhdh/scripts/prepare-restricted-environment.sh \
    --to-registry registry.example.com

  # Extract all the images needed into the specified directory.
  .rhdh/scripts/prepare-restricted-environment.sh \
    --to-dir  /path/to/my/dir

  # From a bastion host connected to the disconnected network,
  # install the operator by using the images from the specified directory.
  .rhdh/scripts/prepare-restricted-environment.sh \
    --from-dir  /path/to/my/dir \
    --to-registry registry.example.com

Copy link

openshift-ci bot commented Jan 27, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from rm3l. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rm3l rm3l force-pushed the RHIDP-4415--airgap-install-script-improvements branch from 9ec9224 to 71d15f5 Compare January 30, 2025 10:07
@rm3l rm3l force-pushed the RHIDP-4415--airgap-install-script-improvements branch from d15754f to 71379e2 Compare February 4, 2025 14:36
@rm3l rm3l force-pushed the RHIDP-4415--airgap-install-script-improvements branch from 71379e2 to d9b7010 Compare February 4, 2025 14:48
@rm3l rm3l force-pushed the RHIDP-4415--airgap-install-script-improvements branch from d9b7010 to 6f15425 Compare February 4, 2025 14:52
@rm3l rm3l changed the title [WIP] Improve the airgap env preparation script [RHIDP-5230] chore: Improve the airgap env preparation script [RHIDP-5230] Feb 4, 2025
@rm3l rm3l marked this pull request as ready for review February 4, 2025 23:33
@openshift-ci openshift-ci bot requested review from kadel and nickboldt February 4, 2025 23:33
@rm3l
Copy link
Member Author

rm3l commented Feb 5, 2025

/cc @zdrapela

@openshift-ci openshift-ci bot requested a review from zdrapela February 5, 2025 08:49
@rm3l rm3l changed the title chore: Improve the airgap env preparation script [RHIDP-5230] chore: Improve the airgap env preparation script [RHIDP-1442][RHIDP-4415] Feb 5, 2025
rm3l and others added 18 commits February 7, 2025 15:17
Image names should follow this format: <project>/<name>
…P registry

It works by adding the pull secrets to the namespace default service account. So it will only work when creating the CR in the operator namespace.
Otherwise, we are displaying the instructions for different namespaces
This is useful if users want to explicitly use `oc-mirror` as their
tool of choice and they know they are already on OCP

TBD: this currently does not work with the internal OCP registry when autodetected by the script
Otherwise tools like 'skopeo login' will attempt to write to /run,
which might be restricted in CI environments.
This also ensures that the credentials don't conflict with
any existing creds for the same registry.

Co-authored-by: Zbynek Drapela <[email protected]>
Otherwise tools like 'skopeo login' will attempt to write to /run,
which might be restricted in CI environments.

Since the user is required to be logged into the index image registry (and the target mirror registry eventually),
it also makes sure these auth information are not lost when switching to a temporary auth file.
Otherwise, as depicted in [1], this might cause some 403 errors to be returned to Skopeo.

This fixes the behavior seen on the QE airgap bastion host.

[1] https://access.redhat.com/solutions/6022011
@rm3l rm3l force-pushed the RHIDP-4415--airgap-install-script-improvements branch from c6dc186 to 13b5e0a Compare February 7, 2025 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants