sap_ha_pacemaker_cluster: Support for VMWare and for ERS2

[rob0d]

Hi all,

Has anyone tried using this to configure ASCS/ERS cluster on VMWare?
I guess the main difference will be the fencing agent, but if anyone did this already it would be really helpful if you can share your experience.

Also the samples and comments in code seem to imply that I can install a combined ASCS/ERS cluster rather than a separate ERS2 cluster. Again, anybody has any view/experience? I need to install two separate clusters. Two node ASCS and two node ERS2.

Any pointers/advice will be greatly appreciated.

[sean-freeman]

@rob0d There is no vars or platform handlers added in the sap_ha_pacemaker_cluster Ansible Role to handle VMware, at this time. The Ansible Role combined with a VMware VM target has been tested by a customer, by using custom variables, but not added into upstream.

The main prohibitor to the aforementioned effort, is the lack of end-to-end flow because of VMware problems that require a deep VMware expert to address. I've got plenty of VMware SDDCs I can access, just cannot get a reliable provision so it breaks automated testing. See:

• notes regarding cloud-init for VMware in community.sap_infrastructure/roles/sap_vm_provision/PLATFORM_GUIDANCE.md
• related GH Issue in vmware.vmware_rest Ansible Collection

The latter question, I will have to defer to @ja9fuchs and @marcelmamula . Would suggest 1 topic per GH Issue/Discussion in future, for ease of discussion

[rob0d]

@sean-freeman

Is this something that you'd be able to share (at least partly) so I don't have to re-invent the wheel?

The Ansible Role combined with a VMware VM target has been tested by a customer, by using custom variables, but not added into upstream.

It seems that these two are more related to VM provisioning, rather than SAP HA install on VMWare?

The main prohibitor to the aforementioned effort, is the lack of end-to-end flow because of VMware problems that require a deep VMware expert to address. I've got plenty of VMware SDDCs I can access, just cannot get a reliable provision so it breaks automated testing. See:

• notes regarding cloud-init for VMware in community.sap_infrastructure/roles/sap_vm_provision/PLATFORM_GUIDANCE.md
• related GH Issue in vmware.vmware_rest Ansible Collection

If I manage the SAP HA part working, do you want me to do a PR to see if we can integrate it back to upstream?

[sean-freeman]

Re. reinvent wheel =
Not for a few weeks, other priorities are calling.

Re. Do-it-faster =
To implement a new default is not that much effort thankfully, just patience required getting the STONITH with fence_vmware_rest Fence Agent parameters correct and thereafter copy/paste/tweak of the IPaddr2 Resource Agent configuration seen elsewhere in the Ansible Role. Thankfully our friends at OVH (thanks @remrozk) already have a very concise version you can follow > https://help.ovhcloud.com/csm/en-gb-sap-hana-configure-suse-cluster?id=kb_article_view&sysparm_article=KB0059460#stonith-resource

Re. provisioning VMware VM =
These issues with cloud-init are provisioning blockers, which is what results in "lack of end-to-end flow" so we cannot perform interative testing during HA development and thereafter regression testing.

[sean-freeman]

@rob0d If you do it, note that you will almost certainly need to use the fence_vmware_rest --filter with the names of each node in the HA cluster.

Unfortunately all of the documentation in the fence_vmware_rest code has been moved and is harder to find, this appears to be the documentation about the API endpoint and using the filters:

https://developer.broadcom.com/xapis/vsphere-automation-api/8.0.3/vcenter/api/vcenter/vm/get/

[marcelmamula]

@rob0d Installation of ASCS/ERS cluster is done following SAP documentation and our certifications. ASCS and ERS is meant to run in one cluster, not spread across independent clusters.

You could potentially:

1. Install ASCS / ERS using sap_swpm role across all those 4 nodes.
2. Setup cluster using directly https://github.com/linux-system-roles/ha_cluster , which would mean you need to prepare all inputs that are now taken care of by sap_ha_pacemaker_cluster role. It means run it once for 2 ASCS nodes, second time for 2 ERS nodes.
3. Execute all post steps manually.

Disclaimer: We do not have any documentation or support for such cluster scenario.

[rob0d]

Hi @marcelmamula,
Combined ASCS/ERS cluster is based on the previous (ENSA1) enqueue server which needed to co-locate ASCS with ERS after a failure. Looking at the current RHEL guides (sorry not seen the SuSe guides), they are still using this setup, but don't create the colocation constraint.
However, it seems that SAP are moving into direction of two separate clusters as per this or 3+ node cluster as per this. I'm not sure how much are they going to push this going forward or if the standard combined ASCS/ERS cluster will remain the default option.

[marcelmamula]

Thank you for providing document link @rob0d Documentation you provided looks like SWPM specific and it lacks any HA Cluster information, including fact that Failover (switchover) groups are not concept in Pacemaker.

SAP ENSA2 clusters are certified only for 2 nodes by default and 3 nodes for special cases described in documentation https://documentation.suse.com/sbp/sap-15/html/SAP-S4HA10-setupguide-simplemount-sle15/index.html#multicluster
HA scenario you shared is not certified by SAP partners, so I cannot comment on validity of that document.

[rob0d]

Hi @marcelmamula,
Yes the links I posted are fairly generic, rather than linked to a specific HA solution. They just show that SAP support multi-node cluster or two two-node clusters. However, I haven't realised that the HA solution certification is strict and doesn't necessarily cover different variant of the cluster setup.
It seems that Redhat are supporting multi-node cluster now.
Thanks for providing the link for Suse documentation, I found a few useful tip there.
It seems that I need to re-think my setup and go back to one two-node cluster.
Thanks for getting back to me.