New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ospdo db migration docs #715

Open

pinikomarov wants to merge 14 commits into openstack-k8s-operators:main from pinikomarov:ospdo_db_migration_docs

+131 −12

Contributor

pinikomarov commented Nov 11, 2024

add db migration mysql client setup diffs

Related patches :
#708

Jira: https://issues.redhat.com/browse/OSPRH-6618


          add db migration mysql client setup diffs

705db70

pinikomarov mentioned this pull request

ospdo docs - setup and backend services #708

Open

pinikomarov added 5 commits

November 11, 2024 16:02


          set conditional params for ospdo pods

ddf1ead


          set conditional params for ospdo pods

41dabad


          mariadb upload ns update


          add ovn ospdo vars

974f143


          add dataplane ospdo vars

aeb9876

pinikomarov mentioned this pull request

ospdo scaledown docs #730

Open

pinikomarov marked this pull request as ready for review

November 14, 2024 12:21

pinikomarov added 6 commits

November 17, 2024 10:54


          add db migration mysql client setup diffs


          set conditional params for ospdo pods

8994fcc


          set conditional params for ospdo pods

7cff789


          mariadb upload ns update

32c1010


          add ovn ospdo vars

9eb2d6b


          add dataplane ospdo vars

3d92004

pinikomarov force-pushed the ospdo_db_migration_docs branch from aeb9876 to 3d92004 Compare

November 17, 2024 08:54


          Merge branch 'ospdo_db_migration_docs' of https://github.com/pinikoma…

f0eddcc

…rov/data-plane-adoption_pkomarov into ospdo_db_migration_docs

klgill reviewed

View reviewed changes

docs_user/assemblies/assembly_migrating-databases-to-the-control-plane.adoc Outdated

		@@ -6,6 +6,18 @@ ifdef::context[:parent-context: {context}]

		= Migrating databases to the control plane

		// Set OSPdO different values to be used in same commands

Contributor

klgill Nov 18, 2024

@pinikomarov Is this text a prerequisite that OSPdO customers need to do before migrating databases to the control plane?
And why is this text hidden?

Contributor Author

pinikomarov Nov 19, 2024

this was a comment just to explain the ifeval conditionals below it can be dropped, unless needed as an logic explanation

klgill reviewed

View reviewed changes

docs_user/assemblies/assembly_migrating-databases-to-the-control-plane.adoc Outdated

               include::../modules/proc_stopping-openstack-services.adoc[leveloffset=+1]
-              include::../modules/proc_migrating-databases-to-mariadb-instances.adoc[leveloffset=+1]
+              include::../modules/proc_migrating-databases-to-mariadb-instances.adoc[@, leveloffset=+1]

Contributor

klgill Nov 18, 2024

What is this symbol for?
Same question on the link below.

Contributor Author

pinikomarov Nov 19, 2024

an error on my part I thought it was needed, so the vars pass to the included files, I'll remove that

klgill reviewed

View reviewed changes

docs_user/modules/proc_adopting-compute-services-to-the-data-plane.adoc Outdated

@@ @@ -126,6 +126,9 @@ endif::[] @@
               ifeval::["{build}" == "downstream"]
               $(cat <path_to_SSH_key> | base64 | sed \'s/^/        /')
               endif::[]
+              ifeval::["{OpenStackPreviousInstaller}" == "director_operator"]
+                    $(oc exec -n ${OSPDO_NAMESPACE} -t openstackclient openstackclient -- cat /home/cloud-admin/.ssh/id_rsa | base64 | sed 's/^/        /')

Contributor

klgill Nov 18, 2024

Suggested change

      
                  $(oc exec -n ${OSPDO_NAMESPACE} -t openstackclient openstackclient -- cat /home/cloud-admin/.ssh/id_rsa | base64 | sed 's/^/        /')
          
                  $(oc exec -n $<ospdo_namespace> -t openstackclient openstackclient -- cat /home/cloud-admin/.ssh/id_rsa | base64 | sed 's/^/        /')

Customers will need to replace <ospdo_namespace> with their own namespace, right?

Contributor Author

pinikomarov Nov 19, 2024

right

klgill reviewed

View reviewed changes

docs_user/modules/proc_migrating-databases-to-mariadb-instances.adoc

		$ oc rsh mariadb-copy-data << EOF
		$ oc rsh -n {pod_ns} mariadb-copy-data << EOF

Contributor

klgill Nov 18, 2024

Is oc rsh -n {pod_ns} mariadb-copy-data << EOF supposed to be specific to OSPdO only? Shouldn't oc rsh mariadb-copy-data << EOF still remain for non-OSPdO deployments?

Contributor Author

pinikomarov Nov 19, 2024

the usage of -n namespace should not cause any difference in regular adoptions, since that's the default namespace users can use or omit that.
I'm adding and making the namespace a variables here to make as little changes as possible to the docs, so that when using ospdo it's explicit where we're running actions on.

klgill reviewed

View reviewed changes

docs_user/modules/proc_migrating-ovn-data.adoc Outdated

Comment on lines 36 to 40

+              +
+              ifeval::["{OpenStackPreviousInstaller}" == "director_operator"]
+              get the ocp master holding the osp controller:
+              oc get vmi -n ${ospdo_namespace} -o jsonpath='{.items[0].metadata.labels.kubevirt\.io/nodeName}'
+              endif::[]

Contributor

klgill Nov 18, 2024

Suggested change

      
            +
          
            ifeval::["{OpenStackPreviousInstaller}" == "director_operator"]
          
            get the ocp master holding the osp controller:
          
            oc get vmi -n ${ospdo_namespace} -o jsonpath='{.items[0].metadata.labels.kubevirt\.io/nodeName}'
          
            endif::[]
          
            ifeval::["{OpenStackPreviousInstaller}" == "director_operator"]
          
            . Get the {OpenShiftShort} master node that contains the {OpenStackShort} Controller:
          
            +
          
            ----
          
            $ oc get vmi -n $<ospdo_namespace> -o jsonpath='{.items[0].metadata.labels.kubevirt\.io/nodeName}'
          
            ----
          
            +
          
            * Replace `<ospdo_namespace>` with your OSPdO namespace.
          
            endif::[]

Contributor Author

pinikomarov Nov 19, 2024

ok

klgill reviewed

View reviewed changes

docs_user/modules/proc_migrating-ovn-data.adoc

@@ @@ -44,7 +48,7 @@ apiVersion: cert-manager.io/v1 @@
               kind: Certificate
               metadata:
                 name: ovn-data-cert
-                namespace: openstack
+                namespace: {pod_ns}

Contributor

klgill Nov 18, 2024

pod_ns = pod namespace?
Is this an OSPdO-specific change, or one that applies to a typical deployment?

Contributor Author

pinikomarov Nov 19, 2024

the use of namespace is to be explcit in our actions , so that in ospdo adoption it's clear where (rhoso or ospdo) we're running the pods and commands against.
In regular adoption it can be used and it would make either no change from the default namespace used by rhoso, or explicitly define that the user is running in the rhoso namespace, so that param only adds specificity not error if used, as in rhoso the default namespace is used anyway whether the user specifies it or not

klgill reviewed

View reviewed changes

docs_user/modules/proc_migrating-ovn-data.adoc Outdated

                   k8s.v1.cni.cncf.io/networks: internalapi
+              endif::[]
+              ifeval::["{OpenStackPreviousInstaller}" == "director_operator"]
+                '[{"name": "internalapi-static", "namespace": ${OSPDO_NAMESPACE}, "ips": ["<internalapi-static-ips>"]}]'

Contributor

klgill Nov 18, 2024

Suggested change

      
              '[{"name": "internalapi-static", "namespace": ${OSPDO_NAMESPACE}, "ips": ["<internalapi-static-ips>"]}]'
          
              '[{"name": "internalapi-static", "namespace": $<ospdo_namespace>, "ips": ["<internalapi-static-ips>"]}]'

Contributor Author

pinikomarov Nov 19, 2024

ok

klgill reviewed

View reviewed changes

docs_user/modules/proc_migrating-ovn-data.adoc

@@ @@ -69,10 +74,19 @@ metadata: @@
                 name: ovn-copy-data
                 annotations:
                   openshift.io/scc: anyuid
+              ifeval::["{OpenStackPreviousInstaller}" != "director_operator"]
                   k8s.v1.cni.cncf.io/networks: internalapi

Contributor

klgill Nov 18, 2024 •

edited

Loading

Line 78 only applies to OSPdO?

Contributor Author

pinikomarov Nov 19, 2024

yes, meaning that line:
'[{"name": "internalapi-static", "namespace": ${OSPDO_NAMESPACE}, "ips": [""]}]'
is only used when run in an OSPdO adoption scenario and is specific to access resources in ospdo env.

klgill reviewed

View reviewed changes

docs_user/modules/proc_migrating-ovn-data.adoc

		@@ -103,7 +117,7 @@ EOF
		. Wait for the pod to be ready:

Contributor

klgill Nov 18, 2024

Suggested change

      
            . Wait for the pod to be ready:
          
            ifeval::["{OpenStackPreviousInstaller}" == "director_operator"]
          
            +
          
            * Replace `<ocp_node_holding_controller>` with the {OpenStackShort} node that contains the Controller.
          
            endif::[]
          
            . Wait for the pod to be ready:

Contributor Author

pinikomarov Nov 19, 2024

ok

klgill reviewed

View reviewed changes

docs_user/modules/proc_migrating-ovn-data.adoc Outdated

@@ @@ -103,7 +117,7 @@ EOF @@
               . Wait for the pod to be ready:
               +
               ----
-              $ oc wait --for=condition=Ready pod/ovn-copy-data --timeout=30s
+              $ oc wait --for=condition=Ready -n{pod_ns} pod/ovn-copy-data --timeout=30s

Contributor

klgill Nov 18, 2024

Suggested change

      
            $ oc wait --for=condition=Ready -n{pod_ns} pod/ovn-copy-data --timeout=30s
          
            $ oc wait --for=condition=Ready -n {pod_ns} pod/ovn-copy-data --timeout=30s

Contributor Author

pinikomarov Nov 19, 2024

ok

klgill reviewed

View reviewed changes

docs_user/modules/proc_migrating-ovn-data.adoc Outdated

Comment on lines 210 to 211

		$ oc exec -n ${RHOSO_NAMESPACE} ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_NB_IP:6641 < /backup/ovs-nb.db"
		$ oc exec -n ${RHOSO_NAMESPACE} ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_SB_IP:6642 < /backup/ovs-sb.db"

Contributor

klgill Nov 18, 2024

Suggested change

      
            $ oc exec -n ${RHOSO_NAMESPACE} ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_NB_IP:6641 < /backup/ovs-nb.db"
          
            $ oc exec -n ${RHOSO_NAMESPACE} ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_SB_IP:6642 < /backup/ovs-sb.db"
          
            $ oc exec -n $<rhoso_namespace> ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_NB_IP:6641 < /backup/ovs-nb.db"
          
            $ oc exec -n $<rhoso_namespace> ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_SB_IP:6642 < /backup/ovs-sb.db"

Contributor Author

pinikomarov Nov 19, 2024

ok

klgill reviewed

View reviewed changes

docs_user/modules/proc_migrating-ovn-data.adoc

-              $ oc exec ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_NB_IP:6641 < /backup/ovs-nb.db"
-              $ oc exec ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_SB_IP:6642 < /backup/ovs-sb.db"
+              $ oc exec -n ${RHOSO_NAMESPACE} ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_NB_IP:6641 < /backup/ovs-nb.db"
+              $ oc exec -n ${RHOSO_NAMESPACE} ovn-copy-data -- bash -c "ovsdb-client restore tcp:$PODIFIED_OVSDB_SB_IP:6642 < /backup/ovs-sb.db"
               ----

Contributor

klgill Nov 18, 2024

Suggested change

      
            +
          
            * Replace `<rhoso_namespace>` with the namespace of your {rhos_acro} deployment.

klgill reviewed

View reviewed changes

docs_user/modules/proc_retrieving-topology-specific-service-configuration.adoc Outdated

Comment on lines 12 to 14

+              // Set OSPdO different values to be used in same commands
+              // When there is an OSPdO source env, some commands run on RHOSO namespace
+              // and some on the OSPdO namespace with some added, different parameteres

Contributor

klgill Nov 18, 2024

Why is this text hidden?

Contributor Author

pinikomarov Nov 19, 2024

just an explanation about the logic below it , It can be removed

klgill reviewed

View reviewed changes

docs_user/modules/proc_retrieving-topology-specific-service-configuration.adoc

               . Export the shell variables for the following outputs and test the connection to the {OpenStackShort} database:
               +
               ----
-              export PULL_OPENSTACK_CONFIGURATION_DATABASES=$(oc run mariadb-client ${MARIADB_CLIENT_ANNOTATIONS} -q --image ${MARIADB_IMAGE} -i --rm --restart=Never -- \
+              export PULL_OPENSTACK_CONFIGURATION_DATABASES=$(oc run mariadb-client {pod_annotations} -q --image ${MARIADB_IMAGE} -i --rm --restart=Never -- \

Contributor

klgill Nov 18, 2024

{pod_annotations} applies to all deployments, not just OSPdO?

Contributor Author

pinikomarov Nov 19, 2024

just to ospdo deployments , as the ifeval above states. the rhoso deployents should have this text(literal): ${MARIADB_CLIENT_ANNOTATIONS}

klgill reviewed

View reviewed changes

docs_user/modules/proc_retrieving-topology-specific-service-configuration.adoc Outdated

+              CONTROLLER1_SSH="oc -n $OSPDO_NAMESPACE rsh -c openstackclient openstackclient ssh controller-0.ctlplane"
+              export CONTROLLER1_SSH
+              ----
+              * With OSPdO, the mariadb-client needs to run on the same OCP node where the OSP Controller is running.  In addition, the internalapi-static network needs to be attached to the pod.

Contributor

klgill Nov 18, 2024 •

edited

Loading

Suggested change

      
            * With OSPdO, the mariadb-client needs to run on the same OCP node where the OSP Controller is running.  In addition, the internalapi-static network needs to be attached to the pod.
          
            * With OSPdO, the `mariadb-client` needs to run on the same {rhocp_long} node where the {OpenStackShort} Controller node is running. In addition, the `internalapi-static` network needs to be attached to the pod.

Contributor Author

pinikomarov Nov 19, 2024

ok

klgill reviewed

View reviewed changes

docs_user/modules/proc_retrieving-topology-specific-service-configuration.adoc Outdated

Comment on lines 72 to 75

+              . Get the passwords file
+              ----
+              oc get secret tripleo-passwords -n $OSPDO_NAMESPACE -o json | jq -r '.data["tripleo-overcloud-passwords.yaml"]' |
+              base64 -d >"${PASSWORD_FILE}"

Contributor

klgill Nov 18, 2024

Suggested change

      
            . Get the passwords file
          
            ----
          
            oc get secret tripleo-passwords -n $OSPDO_NAMESPACE -o json | jq -r '.data["tripleo-overcloud-passwords.yaml"]' |
          
            base64 -d >"${PASSWORD_FILE}"
          
            . Get the passwords file:
          
            +
          
            ----
          
            $ oc get secret tripleo-passwords -n $OSPDO_NAMESPACE -o json | jq -r '.data["tripleo-overcloud-passwords.yaml"]' |
          
            base64 -d >"${PASSWORD_FILE}"

Contributor Author

pinikomarov Nov 19, 2024

ok

klgill reviewed

View reviewed changes

docs_user/modules/proc_retrieving-topology-specific-service-configuration.adoc Outdated

Comment on lines 76 to 89

+              ----
+              . Get the name of the OCP node where the OSP Controller VM is running
+              ----
+              CONTROLLER_NODE=$(oc get vmi -ojson | jq -r '.items[0].status.nodeName')
+              export CONTROLLER_NODE
+              SOURCE_OVN_OVSDB_IP=172.17.0.160 # TODO - get this from the source OVN DB
+              export SOURCE_OVN_OVSDB_IP
+              SOURCE_DB_ROOT_PASSWORD=$(grep <"${PASSWORD_FILE}" ' MysqlRootPassword:' | awk -F ': ' '{ print $2; }') || {
+                  echo "Failed to get the source DB root password"
+                  exit 1
+              }
+              export SOURCE_DB_ROOT_PASSWORD
+              ----

Contributor

klgill Nov 18, 2024

Suggested change

      
            ----
          
            . Get the name of the OCP node where the OSP Controller VM is running
          
            ----
          
            CONTROLLER_NODE=$(oc get vmi -ojson | jq -r '.items[0].status.nodeName')
          
            export CONTROLLER_NODE
          
            SOURCE_OVN_OVSDB_IP=172.17.0.160 # TODO - get this from the source OVN DB
          
            export SOURCE_OVN_OVSDB_IP
          
            SOURCE_DB_ROOT_PASSWORD=$(grep <"${PASSWORD_FILE}" ' MysqlRootPassword:' | awk -F ': ' '{ print $2; }') || {
          
                echo "Failed to get the source DB root password"
          
                exit 1
          
            }
          
            export SOURCE_DB_ROOT_PASSWORD
          
            ----
          
            ----
          
            . Get the name of the {OpenShiftShort} node where the {OpenStackShort} Controller virtual machine is running:
          
            +
          
            ----
          
            $ CONTROLLER_NODE=$(oc get vmi -ojson | jq -r '.items[0].status.nodeName')
          
            export CONTROLLER_NODE
          
            SOURCE_OVN_OVSDB_IP=172.17.0.160 # TODO - get this from the source OVN DB
          
            export SOURCE_OVN_OVSDB_IP
          
            SOURCE_DB_ROOT_PASSWORD=$(grep <"${PASSWORD_FILE}" ' MysqlRootPassword:' | awk -F ': ' '{ print $2; }') || {
          
                echo "Failed to get the source DB root password"
          
                exit 1
          
            }
          
            export SOURCE_DB_ROOT_PASSWORD
          
            ----

Contributor Author

pinikomarov Nov 19, 2024

ok

klgill reviewed

View reviewed changes

docs_user/modules/proc_retrieving-topology-specific-service-configuration.adoc Outdated

Comment on lines 90 to 92

+              . In OSPdO, the mysql service iP can be found in the tripleo-exports-default ConfigMap, section ctlplane-export.yaml
+              ----
+              cpexport=$(oc -n "${OSPDO_NAMESPACE}" get cm tripleo-exports-default -o json | jq -r '.data["ctlplane-export.yaml"]')

Contributor

klgill Nov 18, 2024

Suggested change

      
            . In OSPdO, the mysql service iP can be found in the tripleo-exports-default ConfigMap, section ctlplane-export.yaml
          
            ----
          
            cpexport=$(oc -n "${OSPDO_NAMESPACE}" get cm tripleo-exports-default -o json | jq -r '.data["ctlplane-export.yaml"]')
          
            . Find the mysql service IP in the `ctlplane-export.yaml` section of the `tripleo-exports-default` ConfigMap:
          
            +
          
            ----
          
            $ cpexport=$(oc -n "${OSPDO_NAMESPACE}" get cm tripleo-exports-default -o json | jq -r '.data["ctlplane-export.yaml"]')

Contributor Author

pinikomarov Nov 19, 2024

ok

klgill reviewed

View reviewed changes

docs_user/modules/proc_retrieving-topology-specific-service-configuration.adoc Outdated

+              . Export the shell variables for the following outputs and test the connection to the {OpenStackShort} database:
+              +
+              ----
+              PULL_OPENSTACK_CONFIGURATION_DATABASES="$(oc run mariadb-client -q --image "${MARIADB_IMAGE}" \

Contributor

klgill Nov 18, 2024

Should the "$" precede "PULL_OPENSTACK_CONFIGURATION_DATABASES..."?

Contributor Author

pinikomarov Nov 19, 2024

yes

klgill requested changes

View reviewed changes

Contributor

klgill left a comment

I added suggestions, and there are some questions that need to be addressed.


          updated with review

41eb411

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet