Add blog post for StarlingX dual-stack (IPv4/IIPv6) support

src/pages/author/andre-kantek.md

@@ -0,0 +1,5 @@
+---
+templateKey: 'tags'
+authorName: 'Andre Kantek'
+company: Wind River
+---

src/pages/blog/starlingx-dual-stack-support.md

@@ -0,0 +1,267 @@
+---
+templateKey: blog-post
+title: Dual-Stack (IPv4/IPv6) Support in Starlingx
+author: Andre Kantek
+date: 2024-08-12T01:32:05.627Z
+category:
+  - label: Features & Updates
+    id: category-A7fnZYrE1
+---
+
+StarlingX, a cloud platform designed for edge computing, now supports dual-stack functionality on its platform networks (OAM, management, cluster-host, cluster-pod, cluster-service, admin, storage, and multicast), enabling it to operate with both IPv4 and IPv6 L3 protocols. This enhanced capability offers greater flexibility and scalability for network deployments. Each platform network can be associated to two pools (one IPv4 and another IPv6). The order it is done defines the primary and secondary pool for that network.
+
+The Kubernetes network can also receive dual-stack properties, allowing the pods to use IPv4 and/or IPv6 on its operations. It becomes available when the OAM, cluster-host, cluster-pod, and cluster-service networks are configured as dual-stack.
+
+# Key Features and Considerations
+
+## Address Pool Management
+
+- Two address pools are required for dual-stack operation, with the first one linked to the network at creation.
+- The primary pool cannot be removed, while the secondary pool can be added or removed to transition between single-stack and dual-stack modes.
+- The pxeboot network currently only supports IPv4.
+- The primary address pool is used for internal management communication, as encryption is currently available only for the primary pool on the management network.
+- The primary address pool family of a network cannot be modified after network creation, requiring a system reinstall to change.
+
+## API/CLI Interactions
+
+- The `network-addrpool-assign`, `network-addrpool-remove`, `network-addrpool-list`, and `network-addrpool-show` APIs handle network to address pool associations.
+- The `system network-add` command creates the first association.
+- The `system addrpool-modify` command allows for editing all address pool parameters.
+
+## Installation Bootstrap
+
+- Dual-stack installations require specifying secondary subnets in bootstrap variables using comma-separated values. E.g.:
+  - `external_oam_subnet: fd00::/64,10.20.2.0/24`, the order of network listing determines primary and secondary address pools. In the example fd00::/64 is the primary
+- All primary subnets must use the same address family.
+
+## DNS Server Configuration
+- Optionally, name servers can be configured with both IPv4 and IPv6 addresses using the `system dns-modify` command, still limited to 2 server addresses.
+
+## DC Operations
+- Subclouds can be installed in dual-stack mode if their version supports it.
+- All operational communication between the system controller and subclouds uses the primary address pool.
+- The system-controller and subclouds can operate in different network modes but must share the same primary address family in the OAM and management networks (for subclouds this can be OAM and admin networks).
+- Geo redundancy uses the primary pools to communicate.
+
+## Public Endpoints
+- Public OAM endpoints can be accessed via the secondary address using the same L4 port. HAProxy maps external requests to internal endpoints.
+
+## Network Address Modification at Runtime
+- Only the OAM, Admin, and Management networks can be modified using the `addrpool-modify` command during runtime. 
+- Other network’s address pool modifications require reinstallation.
+
+## External OAM API and CLI
+- The external-OAM API, although deprecated, can still be used to modify the OAM network primary pool. 
+- Corresponding CLIs are `system oam-modify` and `system oam-show`.
+
+## Kubernetes in Dual-Stack
+- The OAM, cluster-host, cluster-service, and cluster-pod networks must be configured for dual-stack support in Kubernetes. 
+- Runtime changes trigger quick restarts for the kube-API-server and kube-controller-manager pods. 
+- New pods will automatically receive both primary and secondary addresses when created in dual-stack mode. Existing pods may retain their current primary addresses and require restarts to acquire secondary addresses. 
+
+## Runtime Configuration
+- To add dual-stack to a running system, follow the steps outlined in the guide, which include adding address pools and associating them with networks.
+- Reverting to single-stack involves removing the network association with the address pool.
+
+# Configuring a Kubernetes network with dual-stack
+This section will show an example on how to use pods that have access to both address families. For more information:
+- https://kubernetes.io/docs/concepts/services-networking/dual-stack/
+- https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/dual-stack-support/
+
+## Bootstrap
+The dual-stack service requires that OAM, cluster-host, cluster-pod, and cluster-service networks be configured with both address families separated by a comma in the bootstrap's localhost file:
+```
+pxeboot_subnet: 192.168.202.0/24 
+external_oam_subnet: fd00::/64,10.20.2.0/24 
+external_oam_gateway_address: fd00::1,10.20.2.1 
+external_oam_floating_address: fd00::3,10.20.2.3 
+external_oam_node_0_address: fd00::4,10.20.2.4 
+external_oam_node_1_address: fd00::5,10.20.2.5 
+management_subnet: fd01::/64 
+management_start_address: fd01::2 
+management_end_address: fd01::ffff 
+cluster_host_subnet: aefd:205::/64,192.168.205.0/24 
+cluster_pod_subnet: aefd:206::/64,172.16.0.0/16 
+cluster_service_subnet: aefd:aaa::/112,10.96.0.0/12 
+```
+
+## Runtime with CLI
+
+This example is considering an AIO-DX installed as IPv6 single-stack.
+
+The list of created networks show the primary pool ID:
+
+```
+# system network-list 
++----+------...--+-----------------+-----------------+---------+-----------...--+---------------------+
+| id | uuid ...  | name            | type            | dynamic | pool_uuid ...  | primary_pool_family |
++----+------...--+-----------------+-----------------+---------+-----------...--+---------------------+
+| 1  | 41eda...e | mgmt            | mgmt            | True    | 0648baa4-2...f | IPv6                |
+| 3  | 484d1...6 | oam             | oam             | False   | cf5f9fe3-d...4 | IPv6                |
+| 4  | 73fd5...0 | multicast       | multicast       | False   | 499fac20-6...e | IPv6                |
+| 2  | 7ce78...5 | pxeboot         | pxeboot         | True    | 765cc97b-b...7 | IPv4                |
+| 5  | 82427...7 | cluster-host    | cluster-host    | True    | 1db96471-f...b | IPv6                |
+| 7  | b3c0a...f | cluster-service | cluster-service | False   | d6318027-3...5 | IPv6                |
+| 6  | f6349...b | cluster-pod     | cluster-pod     | False   | c311c1d6-8...a | IPv6                |
++----+------...--+-----------------+-----------------+---------+-----------...--+---------------------+
+```
+The relationship between address pool and network is also show with `network-addrpool-list`:
+```
+# system network-addrpool-list 
++--------------------------------------+-----------------+-----------------------------+
+| uuid                                 | network_name    | addrpool_name               |
++--------------------------------------+-----------------+-----------------------------+
+| be3afd5d-ef21-495c-b5a7-1494c7b040fc | cluster-host    | cluster-host-subnet-ipv6    |
+| 5530e24c-f868-4cca-9dcc-ba860078a83d | cluster-pod     | cluster-pod-subnet-ipv6     |
+| 536ef1c5-a425-4b94-819a-32849703ec69 | cluster-service | cluster-service-subnet-ipv6 |
+| a061620b-bc27-443b-a0d6-10dafbaae50e | mgmt            | management-ipv6             |
+| 90523d67-6320-4c06-8c45-90dfab4ed3b6 | multicast       | multicast-subnet-ipv6       |
+| 26e5cc6f-cfa8-46f4-8432-8f6cab3e1c82 | oam             | oam-ipv6                    |
+| dbaea2d7-c22a-48ec-83d3-314ae26f7d30 | pxeboot         | pxeboot                     |
++--------------------------------------+-----------------+-----------------------------+
+```
+The actual addresses that will be used by the system can be seen with `addrpool-list`:
+```
+# system addrpool-list 
++------...--+-----------------------------+---------------+--------+--------+---------------------------------------+---...-+
+| uuid ...  | name                        | network       | prefix | order  | ranges                                | fl... |
++------...--+-----------------------------+---------------+--------+--------+---------------------------------------+---...-+
+| 1db96...b | cluster-host-subnet-ipv6    | fd02::        | 64     | random | ['fd02::1-fd02::ffff:ffff:ffff:fffe'] | fd... |
+| c311c...a | cluster-pod-subnet-ipv6     | fd03::        | 64     | random | ['fd03::1-fd03::ffff:ffff:ffff:fffe'] | No... |
+| d6318...5 | cluster-service-subnet-ipv6 | fd04::        | 112    | random | ['fd04::1-fd04::fffe']                | No... |
+| 0648b...f | management-ipv6             | fd01::        | 64     | random | ['fd01::1-fd01::ffff']                | fd... |
+| 499fa...e | multicast-subnet-ipv6       | ff08::1:1:0   | 124    | random | ['ff08::1:1:1-ff08::1:1:e']           | No... |
+| cf5f9...4 | oam-ipv6                    | fd00::        | 64     | random | ['fd00::1-fd00::ffff:ffff:ffff:fffe'] | fd... |
+| 765cc...7 | pxeboot                     | 169.254.202.0 | 24     | random | ['169.254.202.1-169.254.202.254']     | 16... |
++------...--+-----------------------------+---------------+--------+--------+---------------------------------------+---...-+
+```
+The first step is to add the secondary OAM address pool and associate to the network
+```
+system addrpool-add oam-ipv4 10.10.204.0 24 --order random --ranges 10.10.204.1-10.10.204.253 \
+ --floating-address 10.10.204.1 --controller0-address 10.10.204.2 --controller1-address 10.10.204.3 \
+ --gateway-address 10.10.204.254
+
+system network-addrpool-assign oam oam-ipv4  
+```
+Since this system is AIO-DX it will trigger a "Configuration is out-of-date" alarm for both controllers, it will require a lock/unlock cycle.
+```
+system host-lock controller-1
+system host-unlock controller-1
+system host-swact controller-0
+system host-lock controller-0
+system host-unlock controller-0
+system host-swact controller-1
+```
+For AIO-SX systems the step above isn't necessary, as OAM reconfiguration is done at runtime.
+
+The next step is to add the cluster pools and associate them with its networks:
+```
+system addrpool-add cluster-pod-ipv4 172.16.0.0 16 --order random --ranges 172.16.0.1-172.16.254.254 
+system addrpool-add cluster-service-ipv4 10.96.0.0 12 --order random --ranges 10.96.0.1-10.96.254.254
+system addrpool-add cluster-host-ipv4 171.168.204.0 24 --order random --ranges 171.168.204.1-171.168.204.254 \
+ --floating-address 171.168.204.1 --controller0-address 171.168.204.2 --controller1-address 171.168.204.3
+
+system network-addrpool-assign cluster-service cluster-service-ipv4 
+system network-addrpool-assign cluster-pod cluster-pod-ipv4 
+system network-addrpool-assign cluster-host cluster-host-ipv4  
+```
+It will trigger a runtime Kubernetes and Calico configuration to make dual-stack available. This involves a quick restart of the kube-API-server and kube-controller-manager pods on both controllers. Pods that need to use the new cluster-pod network and already exist prior to this configuration will require a restart to gain the new addresses.
+
+To revert back to single-stack, it is necessary to remove the secondary pool from all cluster networks (host, pod, and service).
+```
+DEL=$(system network-addrpool-list | awk '$6 == "cluster-pod-ipv4" { print $2 }') && system network-addrpool-remove $DEL
+DEL=$(system network-addrpool-list | awk '$6 == "cluster-service-ipv4" { print $2 }') && system network-addrpool-remove $DEL
+DEL=$(system network-addrpool-list | awk '$6 == "cluster-host-ipv4" { print $2 }') && system network-addrpool-remove $DEL
+```
+It also involves a quick restart of kube-API-server and kube-controller-manager and the existing user pods need to be restarted.
+
+## Kubernetes deployment example
+
+A small sample to create a deployment to use the dual-stack configuration, there are no special parameters
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+    app: dualstackpod
+  name: dualstackpod
+spec:
+  replicas: 2
+  selector:
+    matchLabels:
+      app: dualstackpod
+  template:
+    metadata:
+      labels:
+        app: dualstackpod
+    spec:
+      containers:
+      - image: centos/tools
+        imagePullPolicy: IfNotPresent
+        name: dualstackpod
+        command: [ "/bin/bash", "-c", "--" ]
+        args: [ "while true; do sleep 300000; done;" ]
+```
+After pod creation it will be possible to see the pod addresses from both cluster-pod pools:
+```
+# kubectl get pods -o wide 
+NAME                            READY   STATUS    RESTARTS   AGE   IP                          NODE           NOMINATED NODE   READINESS GATES
+dualstackpod-7f9d746fc4-6k55x   1/1     Running   0          25s   fd03::a4ce:fec1:5423:e311   controller-1   <none>           <none>
+dualstackpod-7f9d746fc4-g97jp   1/1     Running   0          25s   fd03::8e22:765f:6121:eb61   controller-0   <none>           <none>
+
+# kubectl exec -it dualstackpod-7f9d746fc4-6k55x -- ip -br -c addr
+lo               UNKNOWN        127.0.0.1/8 ::1/128 
+eth0@if24        UP             172.16.166.129/32 fd03::a4ce:fec1:5423:e311/128 fe80::c4f:9ff:feb2:46cc/64 
+
+# kubectl exec -it dualstackpod-7f9d746fc4-g97jp -- ip -br -c addr
+lo               UNKNOWN        127.0.0.1/8 ::1/128 
+eth0@if18        UP             172.16.192.65/32 fd03::8e22:765f:6121:eb61/128 fe80::c024:64ff:fe41:fd3a/64 
+```
+The service requires the dual-stack information:
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+  labels:
+    app: dualstackpod
+  name: dualstackpod-svc
+spec:
+  ipFamilyPolicy: PreferDualStack
+  ipFamilies:
+  - IPv6
+  - IPv4
+  ports:
+  - name: dualstackpod-udp
+    port: 5201
+    protocol: UDP
+    targetPort: 35201
+  - name: dualstackpod-tcp
+    port: 5201
+    protocol: TCP
+    targetPort: 35201
+  selector:
+    app: dualstackpod
+  type: ClusterIP
+```
+It will select 2 addresses (IPv4 and IPv6) to make the L4 ports available:
+```
+# kubectl get service dualstackpod-svc 
+NAME               TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)             AGE
+dualstackpod-svc   ClusterIP   fd04::4627   <none>        5201/UDP,5201/TCP   9m12s
+
+# kubectl get service dualstackpod-svc -o yaml | grep -A8 "clusterIP:"
+  clusterIP: fd04::4627
+  clusterIPs:
+  - fd04::4627
+  - 10.104.245.67
+  internalTrafficPolicy: Cluster
+  ipFamilies:
+  - IPv6
+  - IPv4
+  ipFamilyPolicy: PreferDualStack
+```
+
+# Conclusion
+
+StarlingX's dual-stack support provides enhanced flexibility and scalability for network deployments. By understanding the key features, configuration steps, and considerations outlined in this guide, you can effectively leverage dual-stack capabilities in your StarlingX environment