Skip to content

Commit

Permalink
Docs: deploying Infinispan with Operator
Browse files Browse the repository at this point in the history
  • Loading branch information
pruivo committed Oct 13, 2023
1 parent 2f33cd4 commit 3c88597
Show file tree
Hide file tree
Showing 8 changed files with 359 additions and 23 deletions.
53 changes: 53 additions & 0 deletions doc/kubernetes/collector/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,56 @@ helm template --debug ${STARTDIR}/../../../provision/minikube/keycloak \
| yq \
> ${BUILDDIR}/helm/keycloak-ispn.yaml


# Infinispan single cluster
helm template --debug ${STARTDIR}/../../../provision/infinispan/ispn-helm \
--set namespace=ispn-namespace \
--set replicas=3 \
--set crossdc.enabled=false \
--set metrics.histograms=false \
--set hotrodPassword="strong-password" \
--set cacheDefaults.crossSiteMode=ASYNC \
--set cacheDefaults.stateTransferMode=AUTO \
> ${BUILDDIR}/helm/ispn-single.yaml

# Infinispan site A deployment
helm template --debug ${STARTDIR}/../../../provision/infinispan/ispn-helm \
--set namespace=ispn-namespace \
--set replicas=3 \
--set crossdc.enabled=true \
--set crossdc.local.name=site-a \
--set crossdc.local.gossipRouterEnabled=true \
--set crossdc.remote.name=site-b \
--set crossdc.remote.gossipRouterEnabled=true \
--set crossdc.remote.namespace=ispn-namespace \
--set crossdc.remote.url=openshift://api.site-b \
--set crossdc.remote.secret=xsite-token-secret \
--set crossdc.route.enabled=true \
--set crossdc.route.tls.keystore.secret=xsite-keystore-secret \
--set crossdc.route.tls.truststore.secret=xsite-truststore-secret \
--set metrics.histograms=false \
--set hotrodPassword="strong-password" \
--set cacheDefaults.crossSiteMode=ASYNC \
--set cacheDefaults.stateTransferMode=AUTO \
> ${BUILDDIR}/helm/ispn-site-a.yaml

# Infinispan site B deployment
helm template --debug ${STARTDIR}/../../../provision/infinispan/ispn-helm \
--set namespace=ispn-namespace \
--set replicas=3 \
--set crossdc.enabled=true \
--set crossdc.local.name=site-b \
--set crossdc.local.gossipRouterEnabled=true \
--set crossdc.remote.name=site-a \
--set crossdc.remote.gossipRouterEnabled=true \
--set crossdc.remote.namespace=ispn-namespace \
--set crossdc.remote.url=openshift://api.site-a \
--set crossdc.remote.secret=xsite-token-secret \
--set crossdc.route.enabled=true \
--set crossdc.route.tls.keystore.secret=xsite-keystore-secret \
--set crossdc.route.tls.truststore.secret=xsite-truststore-secret \
--set metrics.histograms=false \
--set hotrodPassword="strong-password" \
--set cacheDefaults.crossSiteMode=ASYNC \
--set cacheDefaults.stateTransferMode=AUTO \
> ${BUILDDIR}/helm/ispn-site-b.yaml
2 changes: 2 additions & 0 deletions doc/kubernetes/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
* xref:running/index.adoc[]
** xref:running/keycloak-deployment.adoc[]
** xref:running/keycloak-with-external-infinispan.adoc[]
** xref:running/infinispan-deployment.adoc[]
** xref:running/infinispan-crossdc-deployment.adoc[]
** xref:running/concepts/index.adoc[]
*** xref:running/concepts/database-connections.adoc[]
*** xref:running/concepts/threads.adoc[]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
. Configure the credential to access the Infinispan cluster.
+
This credential must be configured in Keycloak to be able to access the cluster.
By default, the password is generated automatic and stored in a secret.
+
Check https://infinispan.org/docs/infinispan-operator/main/operator.html#configuring-authentication[Configuring Authentication] documentation on how to insert a custom username/password or how to retrieve the generated credential.
+
As an example on how to customize, the following `identities.yaml` file sets the username and password with admin permissions
+
[source,yam]
----
credentials:
- username: developer
password: strong-password
roles:
- admin
----
+
The `identities.yaml` could be set in a secret as follows:
+
.Credential Secret
[source,yaml]
----
include::example$helm/ispn-single.yaml[tag=infinispan-credentials]
----
<1> The `identities.yaml` base64 encoded.
+
Or using the CLI
+
[source,bash]
----
kubectl create secret generic --from-file=identities.yaml connect-secret
----
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
. Install the https://infinispan.org/docs/infinispan-operator/main/operator.html#installation[Infinispan Operator]
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
* OpenShift or Kubernetes cluster running
* Understanding of the https://infinispan.org/docs/infinispan-operator/main/operator.html[Infinispan Operator]
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
= Infinispan Deployment: Multiple Clusters
:description: This describes how to deploy Infinispan with cross-site enabled.
:site-a: Site-A
:site-b: Site-B
:ocp: OpenShift
:operator-docs: https://infinispan.org/docs/infinispan-operator/main/operator.html
:xsite-docs: https://infinispan.org/docs/stable/titles/xsite/xsite.html
:ispn: Infinispan
:ispn-operator: Infinispan Operator
:kc: Keycloak

{description}

== Audience

This guide describes the procedures required to deploy {ispn} in a multiple cluster environment (cross-site).
For simplicity, this guide uses the minimum configuration possible that allows {kc} to be used with an external {ispn}.

This guide assumes two {ocp} clusters named `{site-a}` and `{site-b}`

See xref:running/index.adoc[] for additional guides.

== Prerequisites

include::commons/infinispan-prerequisites.adoc[]

== Procedure

include::commons/infinispan-install-operator.adoc[]
include::commons/infinispan-credentials.adoc[]
+
Those commands are required both {ocp} clusters.
Although we use the same credentials in both {ispn} clusters, they could have been different.
Just adapt your {kc} configuration.

. Create a service account.
+
A service account is required to establish the cross-site connection.
The {ispn-operator} uses it to inspect the network configuration from the remote site and configure the local {ispn} cluster based on it.
+
For more details check {operator-docs}#managed-cross-site-connections_cross-site[Managing Cross-Site Connections] documentation.
+

.. First, create the service account and generate an access token in both {ocp} clusters.
+
.Create the service account in `{site-a}`
[source,bash]
----
oc create sa -n ispn-namespace xsite-sa
oc policy add-role-to-user view -n ispn-namespace -z xsite-sa
oc create token -n ispn-namespace xsite-sa > site-a-token.txt
----
+
.Create the service account in `{site-b}`
[source,bash]
----
oc create sa -n ispn-namespace xsite-sa
oc policy add-role-to-user view -n ispn-namespace -z xsite-sa
oc create token -n ispn-namespace xsite-sa > site-b-token.txt
----
+
.. The next step is to deploy the token from `{site-a}` into `{site-b}` and vice-versa
+
.Deploy `{site-b}` token into `{site-a}`
[source,bash]
----
oc create secret generic -n ispn-namespace xsite-token-secret --from-literal=token="$(cat site-b-token.txt)"
----
+
.Deploy `{site-a}` token into `{site-b}`
[source,bash]
----
oc create secret generic -n ispn-namespace xsite-token-secret --from-literal=token="$(cat site-a-token.txt)"
----

. Create TLS secrets
+
In this guide we use {ocp} Route for the cross-site communication.
It uses SNI extension from TLS to direct the traffic to the correct pods.
To achieve that, it is required to configure JGroups to use TLS socket, which is done by injects a Keystore and Truststore with the correct certificates.
+
Form more information check {operator-docs}#securing-cross-site-connections_cross-site[Securing Cross Site Connections] documentation or this https://developers.redhat.com/learn/openshift/cross-site-and-cross-applications-red-hat-openshift-and-red-hat-data-grid[Red Hat Developer Guide].
+
The Keystore and the Truststore are uploaded in an {ocp} secret.
The secret contains the file content, the password to access it and the type of the store.
+
How to create the certificates and the stores are out of the scope of this guide.
+
To upload the Keystore, use the following command:
+
.Deploy a Keystore
[source,bash]
----
oc -n ispn-namespace create secret generic xsite-keystore-secret \
--from-file=keystore.p12="./certs/keystore.p12" \ # <1>
--from-literal=password=secret \ #<2>
--from-literal=type=pkcs12 #<3>
----
<1> The filename and the path to the Keystore.
<2> The password to access the Keystore.
<3> The Keystore type.
+
To upload the Truststore, use the following command:
+
.Deploy a Truststore
[source,bash]
----
oc -n ispn-namespace create secret generic xsite-truststore-secret \
--from-file=truststore.p12="./certs/truststore.p12" \ # <1>
--from-literal=password=caSecret \ # <2>
--from-literal=type=pkcs12 # <3>
----
<1> The filename and the path to the Truststore.
<2> The password to access the Truststore.
<3> The Truststore type.
+
Keystore and Truststore must be uploaded in both {ocp} clusters.

. Create an {ispn} Cluster with Cross-Site enabled
+
The {operator-docs}#setting-up-xsite[Setting Up Cross-Site] documentation provides all the information on how to create and configure your {ispn} cluster with cross-site enabled, including the previous steps.
+
A basic example is provided in this guide using the credentials, tokens and TLS Keystore/Truststore created by the commands from the previous steps.
+
.The {ispn} CR for `{site-a}`
[source,yaml]
----
include::example$helm/ispn-site-a.yaml[tag=infinispan-crossdc]
----
<1> The cluster name
<2> Allows the cluster to be monitored by Prometheus.
<3> If using a custom credential, configure here the secret name.
<4> The name of the local site, in this case `{site-a}`.
<5> Exposing the cross-site connection using {ocp} Route.
<6> The secret name where the Keystore exists as defined in the previous step.
<7> The alias of the certificate inside the Keystore.
<8> The secret key (filename) of the Keystore as defined in the previous step.
<9> The secret name where the Truststore exists as defined in the previous step.
<10> The Truststore key (filename) of the Keystore as defined in the previous step.
<11> The remote site's name, in this case `{site-b}`.
<12> The namespace of the {ispn} cluster from the remote site.
<13> The {ocp} API URL for the remote site.
<14> The secret with the access toke to authenticate into the remote site.
+
For `{site-b}`, the {ispn} CR looks similar to the above.
Note the differences in point 4, 11 and 13.
+
.The {ispn} CR for `{site-a}`
[source,yaml]
----
include::example$helm/ispn-site-b.yaml[tag=infinispan-crossdc]
----

. Creating the caches for {kc}.
+
{kc} requires the following caches to be present: `sessions`, `actionTokens`, `authenticationSessions`, `offlineSessions`, `clientSessions`, `offlineClientSessions`, `loginFailures`, and `work`.
+
The {ispn} {operator-docs}#creating-caches[Cache CR] allows to deploy the caches in the {ispn} cluster.
Cross-site needs to be enabled per cache as documented by {xsite-docs}[Cross Site Documentation].
The documentation contains more details about the options used by this guide.
In the following example shows the Cache CR for `{site-a}`.
+
.sessions in `{site-a}`
[source,yaml]
----
include::example$helm/ispn-site-a.yaml[tag=infinispan-cache-sessions]
----
<1> The cross-site merge policy, invoked when there is a write-write conflict
<2> The remote site name.
<3> The cross-site communication, in this case ASYNC.
<4> The state transfer mode. In automatic, the state transfer is triggered after a network partition automatically.
+
For `{site-b}`, the Cache CR is similar except in point 2.
+
.session in `{site-b}`
[source,yaml]
----
include::example$helm/ispn-site-b.yaml[tag=infinispan-cache-sessions]
----
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
= Infinispan Deployment: Single Cluster
:description: This describes the Infinispan deployment in a single cluster environment.

{description}

== Audience

This guide describes the procedure required to deploy Infinispan in a single cluster environment.
For simplicity, this guide uses the minimum configuration possible that allows Keycloak to be used with an external infinispan.

See xref:running/index.adoc[] for additional guides.

== Prerequisites

include::commons/infinispan-prerequisites.adoc[]

== Procedure

include::commons/infinispan-install-operator.adoc[]
include::commons/infinispan-credentials.adoc[]

. Create an Infinispan cluster.
+
The https://infinispan.org/docs/infinispan-operator/main/operator.html#creating-clusters[Creating Infinispan clusters] documentation provides all the information on how to create and configure your Infinispan cluster.
+
For simplicity, we provide the following example as a starting point.
+
.Infinispan CR
[source,yaml]
----
include::example$helm/ispn-single.yaml[tag=infinispan-single]
----
<1> The cluster name
<2> Allows the cluster to be monitored by Prometheus
<3> If using a custom credential, configure here the secret name.

. Creating the caches for Keycloak.
+
Keycloak requires the following caches to be present: `sessions`, `actionTokens`, `authenticationSessions`, `offlineSessions`, `clientSessions`, `offlineClientSessions`, `loginFailures`, and `work`.
+
The Infinispan https://infinispan.org/docs/infinispan-operator/main/operator.html#creating-caches[Cache CR] allows to deploy the caches in the Infinispan cluster.
+
There are no restriction on the cache configuration and an example is provided below for `sessions` cache:
+
.Cache CR for sessions cache
[source,yaml]
----
include::example$helm/ispn-single.yaml[tag=infinispan-cache-sessions]
----
+
The https://infinispan.org/docs/stable/titles/configuring/configuring.html[Infinispan documentation] contains more detail about cache configuration ands its options.
Loading

0 comments on commit 3c88597

Please sign in to comment.