Merge pull request #191 from bogdando/nova_adoption_no_ffu

Nova services adoption (no extra cell, no ffu)
openstack-k8s-operators · Nov 17, 2023 · e505c3a · e505c3a
2 parents 2db4918 + 9100c9a
commit e505c3a
Show file tree

Hide file tree

Showing 17 changed files with 392 additions and 102 deletions.
diff --git a/docs/README.md b/docs/README.md
@@ -29,6 +29,8 @@ Perform the actions from the sub-documents in the following order:
 
 * [Placement adoption](openstack/placement_adoption.md)
 
+* [Nova adoption](openstack/nova_adoption.md)
+
 * [Cinder adoption](openstack/cinder_adoption.md)
 
 * [Manila adoption](openstack/manila_adoption.md)

diff --git a/docs/openstack/edpm_adoption.md b/docs/openstack/edpm_adoption.md
@@ -95,55 +95,57 @@ EOF
   $(cat ~/install_yamls/out/edpm/ansibleee-ssh-key-id_rsa | base64 | sed 's/^/        /')
   EOF
   ```
-* Create the Nova Metadata secret (Workaround while nova isn't adopted yet):
+
+* Generate an ssh key-pair `nova-migration-ssh-key` secret
 
   ```bash
-  oc apply -f - <<EOF
-  apiVersion: v1
-  kind: Secret
-  metadata:
-      name: nova-metadata-neutron-config
-  data:
-      05-nova-metadata.conf: |
-  $(echo "[DEFAULT]\nnova_metadata_host = 1.2.3.4\nnova_metadata_port = 8775\nnova_metadata_protocol = http\nmetadata_proxy_shared_secret = 1234567842\n" | base64 | sed 's/^/        /')
-  EOF
+  cd "$(mktemp -d)"
+  ssh-keygen -f ./id -t ed25519 -N ''
+  oc get secret nova-migration-ssh-key || oc create secret generic nova-migration-ssh-key \
+    -n openstack \
+    --from-file=ssh-privatekey=id \
+    --from-file=ssh-publickey=id.pub \
+    --type kubernetes.io/ssh-auth
+  rm -f id*
+  cd -
   ```
 
-* Stop the nova services.
-
-```bash
-
-# Update the services list to be stopped
-
-ServicesToStop=("tripleo_nova_api_cron.service"
-                "tripleo_nova_api.service"
-                "tripleo_nova_compute.service"
-                "tripleo_nova_conductor.service"
-                "tripleo_nova_libvirt.target"
-                "tripleo_nova_metadata.service"
-                "tripleo_nova_migration_target.service"
-                "tripleo_nova_scheduler.service"
-                "tripleo_nova_virtlogd_wrapper.service"
-                "tripleo_nova_virtnodedevd.service"
-                "tripleo_nova_virtproxyd.service"
-                "tripleo_nova_virtqemud.service"
-                "tripleo_nova_virtsecretd.service"
-                "tripleo_nova_virtstoraged.service"
-                "tripleo_nova_vnc_proxy.service")
+* Create a Nova Compute Extra Config service
+    ```yaml
+    oc apply -f - <<EOF
+    apiVersion: v1
+    kind: ConfigMap
+    metadata:
+      name: nova-compute-extraconfig
+      namespace: openstack
+    data:
+      19-nova-compute-cell1-workarounds.conf: |
+        [workarounds]
+        disable_compute_service_check_for_ffu=true
+    ---
+    apiVersion: dataplane.openstack.org/v1beta1
+    kind: OpenStackDataPlaneService
+    metadata:
+      name: nova-compute-extraconfig
+      namespace: openstack
+    spec:
+      label: nova.compute.extraconfig
+      configMaps:
+        - nova-compute-extraconfig
+      secrets:
+        - nova-cell1-compute-config
+        - nova-migration-ssh-key
+      playbook: osp.edpm.nova
+    EOF
+    ```
 
-echo "Stopping nova services"
-
-for service in ${ServicesToStop[*]}; do
-    echo "Stopping the $service in each controller node"
-    $CONTROLLER1_SSH sudo systemctl stop $service
-    $CONTROLLER2_SSH sudo systemctl stop $service
-    $CONTROLLER3_SSH sudo systemctl stop $service
-done
-```
+  The secret ``nova-cell<X>-compute-config`` is auto-generated for each
+  ``cell<X>``. That secret, alongside ``nova-migration-ssh-key``, should
+  always be specified for each custom `OpenStackDataPlaneService` related to Nova.
 
 * Deploy OpenStackDataPlaneNodeSet:
 
-  ```
+  ```yaml
   oc apply -f - <<EOF
   apiVersion: dataplane.openstack.org/v1beta1
   kind: OpenStackDataPlaneNodeSet
@@ -160,6 +162,8 @@ done
       - install-os
       - configure-os
       - run-os
+      - libvirt
+      - nova-compute-extraconfig
       - ovn
     env:
       - name: ANSIBLE_CALLBACKS_ENABLED
@@ -276,7 +280,7 @@ done
 
 * Deploy OpenStackDataPlaneDeployment:
 
-  ```
+  ```yaml
   oc apply -f - <<EOF
   apiVersion: dataplane.openstack.org/v1beta1
   kind: OpenStackDataPlaneDeployment
@@ -302,6 +306,7 @@ done
     ```
 
 * Wait for the dataplane node set to reach the Ready status:
+
     ```
     oc wait --for condition=Ready osdpns/openstack --timeout=30m
     ```
diff --git a/docs/openstack/mariadb_copy.md b/docs/openstack/mariadb_copy.md
@@ -4,6 +4,11 @@ This document describes how to move the databases from the original
 OpenStack deployment to the MariaDB instances in the OpenShift
 cluster.
 
+> **NOTE** This example scenario describes a simple single-cell setup. Real
+> multi-stack topology recommended for production use results in different
+> cells DBs layout, and should be using different naming schemes (not covered
+> here this time).
+
 ## Prerequisites
 
 * Make sure the previous Adoption steps have been performed successfully.
@@ -13,7 +18,7 @@ cluster.
   * Podified MariaDB and RabbitMQ are running. No other podified
     control plane services are running.
 
-  * OpenStack services have been stopped
+  * OpenStack services have been [stopped](stop_openstack_services.md)
 
   * There must be network routability between:
 
@@ -34,7 +39,7 @@ cluster.
 Define the shell variables used in the steps below. The values are
 just illustrative, use values that are correct for your environment:
 
-```
+```bash
 MARIADB_IMAGE=quay.io/podified-antelope-centos9/openstack-mariadb:current-podified
 
 PODIFIED_MARIADB_IP=$(oc get svc --selector "cr=mariadb-openstack" -ojsonpath='{.items[0].spec.clusterIP}')
@@ -57,21 +62,21 @@ COLLATION=utf8_general_ci
 
 * Test connection to the original DB (show databases):
 
-  ```
+  ```bash
   podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE \
       mysql -h "$SOURCE_MARIADB_IP" -uroot "-p$SOURCE_DB_ROOT_PASSWORD" -e 'SHOW databases;'
   ```
 
 * Run mysqlcheck on the original DB to look for things that are not OK:
 
-  ```
+  ```bash
   podman run -i --rm --userns=keep-id -u $UID $MARIADB_IMAGE \
       mysqlcheck --all-databases -h $SOURCE_MARIADB_IP -u root "-p$SOURCE_DB_ROOT_PASSWORD" | grep -v OK
   ```
 
 * Test connection to podified DBs (show databases):
 
-  ```
+  ```bash
   oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \
       mysql -h "$PODIFIED_MARIADB_IP" -uroot "-p$PODIFIED_DB_ROOT_PASSWORD" -e 'SHOW databases;'
   oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \
@@ -80,17 +85,25 @@ COLLATION=utf8_general_ci
 
 ## Procedure - data copy
 
+> **NOTE**: We'll need to transition Nova services imported later on into a
+> superconductor architecture. For that, delete the old service records in
+> cells DBs, starting from the cell1. New records will be registered with
+> different hostnames provided by the Nova service operator. All Nova
+> services, except the compute agent, have no internal state, and its service
+> records can be safely deleted. Also we need to rename the former `default` cell
+> as `cell1`.
+
 * Create a temporary folder to store DB dumps and make sure it's the
   working directory for the following steps:
 
-  ```
+  ```bash
   mkdir ~/adoption-db
   cd ~/adoption-db
   ```
 
 * Create a dump of the original databases:
 
-  ```
+  ```bash
   podman run -i --rm --userns=keep-id -u $UID -v $PWD:$PWD:z,rw -w $PWD $MARIADB_IMAGE bash <<EOF
 
   # Note we do not want to dump the information and performance schema tables so we filter them
@@ -106,7 +119,7 @@ COLLATION=utf8_general_ci
 
 * Restore the databases from .sql files into the podified MariaDB:
 
-  ```
+  ```bash
   # db schemas to rename on import
   declare -A db_name_map
   db_name_map["nova"]="nova_cell1"
@@ -148,13 +161,17 @@ COLLATION=utf8_general_ci
       oc run ${container_name} --image ${MARIADB_IMAGE} -i --rm --restart=Never -- \
           mysql -h "${db_server}" -uroot "-p${db_password}" "${db_name}" < "${db_file}"
   done
+  oc exec -it mariadb-openstack -- mysql --user=root --password=${db_server_password_map["default"]} -e \
+      "update nova_api.cell_mappings set name='cell1' where name='default';"
+  oc exec -it mariadb-openstack-cell1 -- mysql --user=root --password=${db_server_password_map["default"]} -e \
+      "delete from nova_cell1.services where host not like '%nova-cell1-%' and services.binary != 'nova-compute';"
   ```
 
 ## Post-checks
 
 * Check that the databases were imported correctly:
 
-  ```
+  ```bash
   oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \
   mysql -h "${PODIFIED_MARIADB_IP}" -uroot "-p${PODIFIED_DB_ROOT_PASSWORD}" -e 'SHOW databases;' \
       | grep keystone

diff --git a/docs/openstack/nova_adoption.md b/docs/openstack/nova_adoption.md
@@ -0,0 +1,126 @@
+# Nova adoption
+
+> **NOTE** This example scenario describes a simple single-cell setup. Real
+> multi-stack topology recommended for production use results in different
+> cells DBs layout, and should be using different naming schemes (not covered
+> here this time).
+
+## Prerequisites
+
+* Previous Adoption steps completed. Notably,
+  * the [service databases](mariadb_copy.md)
+    must already be imported into the podified MariaDB;
+  * the [Keystone service](keystone_adoption.md) needs to be imported;
+  * the [Placement service](placement_adoption.md) needs to be imported;
+  * the [Glance service](glance_adoption.md) needs to be imported;
+  * the [OVN DB services](ovn_adoption.md) need to be imported;
+  * the [Neutron service](neutron_adoption.md) needs to be imported;
+  * Required services specific topology [configuration collected](pull_openstack_configuration.md#get-services-topology-specific-configuration);
+  * OpenStack services have been [stopped](stop_openstack_services.md)
+
+## Variables
+
+Define the shell variables and aliases used in the steps below. The values are
+just illustrative, use values that are correct for your environment:
+
+```bash
+alias openstack="oc exec -t openstackclient -- openstack"
+```
+
+## Procedure - Nova adoption
+
+> **NOTE**: We assume Nova Metadata deployed on the top level and not on each
+> cell level, so this example imports it the same way. If the source deployment
+> has a per cell metadata deployment, adjust the given below patch as needed.
+> Metadata service cannot be run in `cell0`.
+
+* Patch OpenStackControlPlane to deploy Nova:
+
+  ```yaml
+  oc patch openstackcontrolplane openstack -n openstack --type=merge --patch '
+  spec:
+    nova:
+      enabled: true
+      apiOverride:
+        route: {}
+      template:
+        secret: osp-secret
+        apiServiceTemplate:
+          override:
+            service:
+              internal:
+                metadata:
+                  annotations:
+                    metallb.universe.tf/address-pool: internalapi
+                    metallb.universe.tf/allow-shared-ip: internalapi
+                    metallb.universe.tf/loadBalancerIPs: 172.17.0.80
+                spec:
+                  type: LoadBalancer
+          customServiceConfig: |
+            [workarounds]
+            disable_compute_service_check_for_ffu=true
+        metadataServiceTemplate:
+          enabled: true # deploy single nova metadata on the top level
+          override:
+            service:
+              metadata:
+                annotations:
+                  metallb.universe.tf/address-pool: internalapi
+                  metallb.universe.tf/allow-shared-ip: internalapi
+                  metallb.universe.tf/loadBalancerIPs: 172.17.0.80
+              spec:
+                type: LoadBalancer
+          customServiceConfig: |
+            [workarounds]
+            disable_compute_service_check_for_ffu=true
+        schedulerServiceTemplate:
+          customServiceConfig: |
+            [workarounds]
+            disable_compute_service_check_for_ffu=true
+        cellTemplates:
+          cell0:
+            conductorServiceTemplate:
+              customServiceConfig: |
+                [workarounds]
+                disable_compute_service_check_for_ffu=true
+          cell1:
+            metadataServiceTemplate:
+              enabled: false # enable here to run it in a cell instead
+              override:
+                  service:
+                    metadata:
+                      annotations:
+                        metallb.universe.tf/address-pool: internalapi
+                        metallb.universe.tf/allow-shared-ip: internalapi
+                        metallb.universe.tf/loadBalancerIPs: 172.17.0.80
+                    spec:
+                      type: LoadBalancer
+              customServiceConfig: |
+                [workarounds]
+                disable_compute_service_check_for_ffu=true
+            conductorServiceTemplate:
+              customServiceConfig: |
+                [workarounds]
+                disable_compute_service_check_for_ffu=true
+  '
+  ```
+
+* Wait for Nova control plane services' CRs to become ready:
+
+  ```bash
+  oc wait --for condition=Ready --timeout=300s Nova/nova
+  ```
+
+  The local Conductor services will be started for each cell, while the superconductor runs in `cell0`.
+  Note that ``disable_compute_service_check_for_ffu`` is mandatory for all imported Nova services, until
+  the [external dataplane imported](edpm_adoption.md), and until Nova Compute services fast-forward upgraded.
+
+## Post-checks
+
+* Check that Nova endpoints are defined and pointing to the
+  podified FQDNs and that Nova API responds.
+
+  ```bash
+  openstack endpoint list | grep nova
+  openstack server list
+  ```
diff --git a/docs/openstack/placement_adoption.md b/docs/openstack/placement_adoption.md
@@ -2,8 +2,12 @@
 
 ## Prerequisites
 
-* Previous Adoption steps completed. Notably, the service databases
-  must already be imported into the podified MariaDB.
+* Previous Adoption steps completed. Notably,
+  * the [service databases](mariadb_copy.md)
+    must already be imported into the podified MariaDB.
+  * the [Keystone service](keystone_adoption.md) needs to be imported.
+  * the Memcached operator needs to be deployed (nothing to import for it from
+    the source environment).
 
 ## Variables
 

diff --git a/docs/openstack/pull_openstack_configuration.md b/docs/openstack/pull_openstack_configuration.md
@@ -44,4 +44,4 @@ Once the ansible playbook has been run, you should have into your local director
   ▾ tmp/
     ▾ collect_tripleo_configs/
       ▾ glance/
-```
+```