Multiple SAP systems with sap_ha_pacemaker_cluster

[rob0d]

Hi all,
I have a situation where I need to install multiple clustered SAP instances onto the same cluster. In this case I will have ASCS/ERS on a two node cluster and need to add a clustered Web Dispatcher instance. In the past we had a case of multiple ASCS/ERS resource groups for several SAP systems. Another case could be the "Cost-Optimised" setup with combined ASCS/ERS and HANA cluster.

So there are multiple cases where deploying independent SAP systems onto the same cluster will be helpful. However, the problem is that sap_ha_pacemaker_cluster/rhel-system-roles.ha_cluster will remove an existing cluster config if they are ran twice. This means I can't just prepare two configurations and execute the roles twice.

I am thinking about an enhancement to sap_ha_pacemaker_cluster which would allow us to prepare several configurations and then execute ha_cluster only once.

I don't think that storing things in two YAML or CIB files and then trying to merge them is a good idea (plain text passwords, etc.), so my current plan is to prepare the variables and extend the resource primitives with extra resources/constrains and groups. Possibly by changing the code here to merge __sap_ha_pacemaker_cluster_resource_groups with what's already in ha_cluster_resource_groups.

Also the role currently doesn't support WebDisp clusters, so I can potentially add that too or use ha_cluster directly to create it.

What would you do with this? Any suggestions, comments, recommendations? Is there a better way of achieving the same outcome?

[ja9fuchs]

@rob0d The SAP HA role is written in a way that it inherits ha_cluster role parameters and combines them with the currently possible SAP setup.
To help with an enhanced configuration (and debugging of input into the ha_cluster role), you could enable and amend the following 2 role vars - copied here with their role default values:

sap_ha_pacemaker_cluster_create_config_varfile: false
sap_ha_pacemaker_cluster_create_config_dest: "review_resource_config.yml"

You run the SAP HA role once with the main parts of your desired setup, then the config file should contain all the variables as they are fed into ha_cluster. You can enhance this config and add the resources and other settings not currently provided by the SAP HA role, but supported by ha_cluster, and then re-run it.

This should also work using a dry-run to just create the ha_cluster role input parameters, without making changes, but creating the config output for inspection and re-use.

Using such an enhanced config as a basis should lead to the desired results, as long as there are no conflicting resource IDs used.

In theory anyway, I have not tested it with an advanced resources setup like you plan.

Security remark:
The config output file is disabled by default, because it will store credentials in plain text if there were any in the original configuration values.

Another edit:
You do not need to use the conf varfile, it could just help with the structure.
You can include ha_cluster parameters for your missing pieces and make sure to include those vars when running the SAP HA role. It should include and combine with what the role provides, at least for all definitions for which that makes sense. Some custom values would take precedence, but the lists of resources and such components should stack if it works as intended.

[rob0d]

Hi @ja9fuchs,
Thanks for the information, much appreciated.
Yes, I have sap_ha_pacemaker_cluster_create_config_* variables set and will check it.

The confirmation that I can pre-populate ha_cluster is really helpful. I had a look at the code and it should do exactly what I need.

I can't test it yet as the HW is not ready and I am still looking at VmWare specific variables for the role. In the meantime would you be able to confirm if these assumptions and options are correct?

1. It seems that I can combine different cluster types by adding them to sap_ha_pacemaker_cluster_host_type. So if I wanted to have e.g. ASCS/ERS and HANA clustered in the same cluster, the role should work as is. Is this correct?

2. I could add extra cluster type ( e.g. webdisp). The code is relatively simple as it's basically like a PAS instance without AAS. Once done, I can call the role with:

 sap_ha_pacemaker_cluster_host_type:
     - nwas_abap_ascs_ers
     - webdisp

and it will create the cluster with the correct resources (assuming I get the code right).

3. I can populate ha_cluster, ha_cluster_resource_groups , ha_cluster_resource_primitives, ha_cluster_constraints_location with sensible values and just call the role with nwas_abap_ascs_ers. I am slightly worried about he order or resource groups as in this case webdisp will be first in the dictionary and will be added first to the cluster primitives which means it will start first (I want it to start last). I am not sure if ha_cluster role allows adding primitives while specifying the order (pcs resource group add has --before / --after).

Thanks a lot

[ja9fuchs]

This is quite the complex setting, for which I can not tell out of the box if every bit of it will work as expected.
Most of it should work the way you describe, the variables in 3. should be inherited and either take precedence (ha_cluster) or be combined with further definitions put together by the role.

1. The host type var was made a list to enable such use cases. However, I have not had time yet to do tests of combined scenarios for real.

2. Yes, ideally that works, depending on the code for this new type that needs to be created too. :)

3. Regarding the order of the resources you may be right. The parameter ha_cluster_resource_groups can take attributes. Not sure if that works for the order options, but it's probably worth trying this way.

Please share how it goes during your tests. Whatever does not work right away can certainly be fixed, as long as it can be used for general use cases. :)

Edit: I suggest to try using a dry-run first (-C). I used this a lot with the _config* vars for previewing the parameters in the way the role would feed the variables into the ha_cluster role. This helps a lot with such initial variable experiments.