Skip to content

Commit

Permalink
refact ip stack (#11953)
Browse files Browse the repository at this point in the history
  • Loading branch information
borislitv authored Feb 11, 2025
1 parent c557adf commit a51e7dd
Show file tree
Hide file tree
Showing 64 changed files with 470 additions and 208 deletions.
16 changes: 13 additions & 3 deletions .gitlab-ci/vagrant.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,21 @@
- .cache/pip
policy: pull-push # TODO: change to "pull" when not on main

vagrant_ubuntu20-calico-dual-stack:
vagrant_ubuntu24-calico-dual-stack:
stage: deploy-extended
extends: .vagrant
when: manual
# FIXME: this test if broken (perma-failing)
rules:
- if: $PR_LABELS =~ /.*(ci-extended|ci-full).*/
when: on_success
allow_failure: false

vagrant_ubuntu24-calico-ipv6only-stack:
stage: deploy-extended
extends: .vagrant
rules:
- if: $PR_LABELS =~ /.*(ci-extended|ci-full).*/
when: on_success
allow_failure: false

vagrant_ubuntu20-flannel:
stage: deploy-part1
Expand Down
8 changes: 7 additions & 1 deletion Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -210,14 +210,20 @@ Vagrant.configure("2") do |config|
end

ip = "#{$subnet}.#{i+100}"
ip6 = "#{$subnet_ipv6}::#{i+100}"
node.vm.network :private_network,
:ip => ip,
:libvirt__guest_ipv6 => 'yes',
:libvirt__ipv6_address => "#{$subnet_ipv6}::#{i+100}",
:libvirt__ipv6_address => ip6,
:libvirt__ipv6_prefix => "64",
:libvirt__forward_mode => "none",
:libvirt__dhcp_enabled => false

# libvirt__ipv6_address does not work as intended, the address is obtained with the desired prefix, but auto-generated(like fd3c:b398:698:756:5054:ff:fe48:c61e/64)
# add default route for detect ansible_default_ipv6
# TODO: fix libvirt__ipv6 or use $subnet in shell
config.vm.provision "shell", inline: "ip -6 r a fd3c:b398:698:756::/64 dev eth1;ip -6 r add default via fd3c:b398:0698:0756::1 dev eth1 || true"

# Disable swap for each vm
node.vm.provision "shell", inline: "swapoff -a"

Expand Down
1 change: 1 addition & 0 deletions contrib/terraform/terraform.py
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,7 @@ def openstack_host(resource, module_name):
'access_ip_v4': raw_attrs['access_ip_v4'],
'access_ip_v6': raw_attrs['access_ip_v6'],
'access_ip': raw_attrs['access_ip_v4'],
'access_ip6': raw_attrs['access_ip_v6'],
'ip': raw_attrs['network.0.fixed_ip_v4'],
'flavor': parse_dict(raw_attrs, 'flavor',
sep='_'),
Expand Down
39 changes: 34 additions & 5 deletions docs/ansible/vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,12 @@ Some variables of note include:
* *ansible_default_ipv4.address* - Not Kubespray-specific, but it is used if ip
and access_ip are undefined
* *ip6* - IPv6 address to use for binding services. (host var)
If *enable_dual_stack_networks* is set to ``true`` and *ip6* is defined,
If *ipv6_stack*(*enable_dual_stack_networks* deprecated) is set to ``true`` and *ip6* is defined,
kubelet's ``--node-ip`` and node's ``InternalIP`` will be the combination of *ip* and *ip6*.
Similarly used for ipv6only scheme.
* *access_ip6* - similarly ``access_ip`` but IPv6
* *ansible_default_ipv6.address* - Not Kubespray-specific, but it is used if ip6
and access_ip6 are undefined
* *loadbalancer_apiserver* - If defined, all hosts will connect to this
address instead of localhost for kube_control_planes and kube_control_plane[0] for
kube_nodes. See more details in the
Expand All @@ -52,6 +56,20 @@ Some variables of note include:
`loadbalancer_apiserver`. See more details in the
[HA guide](/docs/operations/ha-mode.md).

## Special network variables

These variables help avoid a large number of if/else constructs throughout the code associated with enabling different network stack.
These variables are used in all templates.
By default, only ipv4_stack is enabled, so it is given priority in dualstack mode.
Don't change these variables if you don't understand what you're doing.

* *main_access_ip* - equal to ``access_ip`` when ipv4_stack is enabled(even in case of dualstack),
and ``access_ip6`` for IPv6 only clusters
* *main_ip* - equal to ``ip`` when ipv4_stack is enabled(even in case of dualstack),
and ``ip6`` for IPv6 only clusters
* *main_access_ips* - list of ``access_ip`` and ``access_ip6`` for dualstack and one corresponding variable for single
* *main_ips* - list of ``ip`` and ``ip6`` for dualstack and one corresponding variable for single

## Cluster variables

Kubernetes needs some parameters in order to get deployed. These are the
Expand Down Expand Up @@ -83,12 +101,18 @@ following default cluster parameters:
(assertion not applicable to calico which doesn't use this as a hard limit, see
[Calico IP block sizes](https://docs.projectcalico.org/reference/resources/ippool#block-sizes)).

* *enable_dual_stack_networks* - Setting this to true will provision both IPv4 and IPv6 networking for pods and services.

* *kube_service_addresses_ipv6* - Subnet for cluster IPv6 IPs (default is ``fd85:ee78:d8a6:8607::1000/116``). Must not overlap with ``kube_pods_subnet_ipv6``.

* *kube_service_subnets* - All service subnets separated by commas (default is a mix of ``kube_service_addresses`` and ``kube_service_addresses_ipv6`` depending on ``ipv4_stack`` and ``ipv6_stacke`` options),
for example ``10.233.0.0/18,fd85:ee78:d8a6:8607::1000/116`` for dual stack(ipv4_stack/ipv6_stack set to `true`).
It is not recommended to change this variable directly.

* *kube_pods_subnet_ipv6* - Subnet for Pod IPv6 IPs (default is ``fd85:ee78:d8a6:8607::1:0000/112``). Must not overlap with ``kube_service_addresses_ipv6``.

* *kube_pods_subnets* - All pods subnets separated by commas (default is a mix of ``kube_pods_subnet`` and ``kube_pod_subnet_ipv6`` depending on ``ipv4_stack`` and ``ipv6_stacke`` options),
for example ``10.233.64.0/18,fd85:ee78:d8a6:8607::1:0000/112`` for dual stack(ipv4_stack/ipv6_stack set to `true`).
It is not recommended to change this variable directly.

* *kube_network_node_prefix_ipv6* - Subnet allocated per-node for pod IPv6 IPs. Remaining bits in ``kube_pods_subnet_ipv6`` dictates how many kube_nodes can be in cluster.

* *skydns_server* - Cluster IP for DNS (default is 10.233.0.3)
Expand Down Expand Up @@ -152,9 +176,14 @@ Note, if cloud providers have any use of the ``10.233.0.0/16``, like instances'
private addresses, make sure to pick another values for ``kube_service_addresses``
and ``kube_pods_subnet``, for example from the ``172.18.0.0/16``.

## Enabling Dual Stack (IPV4 + IPV6) networking
## Enabling Dual Stack (IPV4 + IPV6) or IPV6 only networking

If *enable_dual_stack_networks* is set to ``true``, Dual Stack networking will be enabled in the cluster. This will use the default IPv4 and IPv6 subnets specified in the defaults file in the ``kubespray-defaults`` role, unless overridden of course. The default config will give you room for up to 256 nodes with 126 pods per node, and up to 4096 services.
IPv4 stack enable by *ipv4_stack* is set to ``true``, by default.
IPv6 stack enable by *ipv6_stack* is set to ``false`` by default.
This will use the default IPv4 and IPv6 subnets specified in the defaults file in the ``kubespray-defaults`` role, unless overridden of course. The default config will give you room for up to 256 nodes with 126 pods per node, and up to 4096 services.
Set both variables to ``true`` for Dual Stack mode.
IPv4 has higher priority in Dual Stack mode(e.g. in variables `main_ip`, `main_access_ip` and other).
You can also make IPv6 only clusters with ``false`` in *ipv4_stack*.

## DNS variables

Expand Down
15 changes: 6 additions & 9 deletions inventory/sample/group_vars/k8s_cluster/k8s-cluster.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,27 +97,24 @@ kube_pods_subnet: 10.233.64.0/18
# - kubelet_max_pods: 110
kube_network_node_prefix: 24

# Configure Dual Stack networking (i.e. both IPv4 and IPv6)
enable_dual_stack_networks: false

# Kubernetes internal network for IPv6 services, unused block of space.
# This is only used if enable_dual_stack_networks is set to true
# This is only used if ipv6_stack is set to true
# This provides 4096 IPv6 IPs
kube_service_addresses_ipv6: fd85:ee78:d8a6:8607::1000/116

# Internal network. When used, it will assign IPv6 addresses from this range to individual pods.
# This network must not already be in your network infrastructure!
# This is only used if enable_dual_stack_networks is set to true.
# This is only used if ipv6_stack is set to true.
# This provides room for 256 nodes with 254 pods per node.
kube_pods_subnet_ipv6: fd85:ee78:d8a6:8607::1:0000/112

# IPv6 subnet size allocated to each for pods.
# This is only used if enable_dual_stack_networks is set to true
# This is only used if ipv6_stack is set to true
# This provides room for 254 pods per node.
kube_network_node_prefix_ipv6: 120

# The port the API Server will be listening on.
kube_apiserver_ip: "{{ kube_service_addresses | ansible.utils.ipaddr('net') | ansible.utils.ipaddr(1) | ansible.utils.ipaddr('address') }}"
kube_apiserver_ip: "{{ kube_service_subnets.split(',') | first | ansible.utils.ipaddr('net') | ansible.utils.ipaddr(1) | ansible.utils.ipaddr('address') }}"
kube_apiserver_port: 6443 # (https)

# Kube-proxy proxyMode configuration.
Expand Down Expand Up @@ -215,8 +212,8 @@ resolvconf_mode: host_resolvconf
# Deploy netchecker app to verify DNS resolve as an HTTP service
deploy_netchecker: false
# Ip address of the kubernetes skydns service
skydns_server: "{{ kube_service_addresses | ansible.utils.ipaddr('net') | ansible.utils.ipaddr(3) | ansible.utils.ipaddr('address') }}"
skydns_server_secondary: "{{ kube_service_addresses | ansible.utils.ipaddr('net') | ansible.utils.ipaddr(4) | ansible.utils.ipaddr('address') }}"
skydns_server: "{{ kube_service_subnets.split(',') | first | ansible.utils.ipaddr('net') | ansible.utils.ipaddr(3) | ansible.utils.ipaddr('address') }}"
skydns_server_secondary: "{{ kube_service_subnets.split(',') | first | ansible.utils.ipaddr('net') | ansible.utils.ipaddr(4) | ansible.utils.ipaddr('address') }}"
dns_domain: "{{ cluster_name }}"

## Container runtime
Expand Down
2 changes: 1 addition & 1 deletion inventory/sample/group_vars/k8s_cluster/k8s-net-calico.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ calico_cni_name: k8s-pod-network

# Enables Internet connectivity from containers
# nat_outgoing: true
# nat_outgoing_ipv6: false
# nat_outgoing_ipv6: true

# Enables Calico CNI "host-local" IPAM plugin
# calico_ipam_host_local: true
Expand Down
2 changes: 1 addition & 1 deletion roles/container-engine/containerd/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ enable_cdi: false
# For containerd tracing configuration please check out the official documentation:
# https://github.com/containerd/containerd/blob/main/docs/tracing.md
containerd_tracing_enabled: false
containerd_tracing_endpoint: "0.0.0.0:4317"
containerd_tracing_endpoint: "[::]:4317"
containerd_tracing_protocol: "grpc"
containerd_tracing_sampling_ratio: 1.0
containerd_tracing_service_name: "containerd"
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Requires=cri-dockerd.socket

[Service]
Type=notify
ExecStart={{ bin_dir }}/cri-dockerd --container-runtime-endpoint {{ cri_socket }} --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --network-plugin=cni --pod-cidr={{ kube_pods_subnet }} --pod-infra-container-image={{ pod_infra_image_repo }}:{{ pod_infra_version }} --log-level {{ cri_dockerd_log_level }} {% if enable_dual_stack_networks %}--ipv6-dual-stack=True{% endif %}
ExecStart={{ bin_dir }}/cri-dockerd --container-runtime-endpoint {{ cri_socket }} --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --network-plugin=cni --pod-cidr={{ kube_pods_subnets }} --pod-infra-container-image={{ pod_infra_image_repo }}:{{ pod_infra_version }} --log-level {{ cri_dockerd_log_level }} {% if ipv6_stack %}--ipv6-dual-stack=True{% endif %}

ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
Expand Down
4 changes: 2 additions & 2 deletions roles/etcd/handlers/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

- name: Wait for etcd up
uri:
url: "https://{% if 'etcd' in group_names %}{{ etcd_address }}{% else %}127.0.0.1{% endif %}:2379/health"
url: "https://{% if 'etcd' in group_names %}{{ etcd_address | ansible.utils.ipwrap }}{% else %}127.0.0.1{% endif %}:2379/health"
validate_certs: false
client_cert: "{{ etcd_cert_dir }}/member-{{ inventory_hostname }}.pem"
client_key: "{{ etcd_cert_dir }}/member-{{ inventory_hostname }}-key.pem"
Expand All @@ -39,7 +39,7 @@

- name: Wait for etcd-events up
uri:
url: "https://{% if 'etcd' in group_names %}{{ etcd_address }}{% else %}127.0.0.1{% endif %}:2383/health"
url: "https://{% if 'etcd' in group_names %}{{ etcd_address | ansible.utils.ipwrap }}{% else %}127.0.0.1{% endif %}:2383/health"
validate_certs: false
client_cert: "{{ etcd_cert_dir }}/member-{{ inventory_hostname }}.pem"
client_key: "{{ etcd_cert_dir }}/member-{{ inventory_hostname }}-key.pem"
Expand Down
4 changes: 2 additions & 2 deletions roles/etcd/tasks/configure.yml
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@
ETCDCTL_ENDPOINTS: "{{ etcd_events_access_addresses }}"

- name: Configure | Check if member is in etcd cluster
shell: "{{ bin_dir }}/etcdctl member list | grep -w -q {{ etcd_access_address }}"
shell: "{{ bin_dir }}/etcdctl member list | grep -w -q {{ etcd_access_address | replace('[', '') | replace(']', '') }}"
register: etcd_member_in_cluster
ignore_errors: true # noqa ignore-errors
changed_when: false
Expand All @@ -163,7 +163,7 @@
ETCDCTL_ENDPOINTS: "{{ etcd_access_addresses }}"

- name: Configure | Check if member is in etcd-events cluster
shell: "{{ bin_dir }}/etcdctl member list | grep -w -q {{ etcd_access_address }}"
shell: "{{ bin_dir }}/etcdctl member list | grep -w -q {{ etcd_access_address | replace('[', '') | replace(']', '') }}"
register: etcd_events_member_in_cluster
ignore_errors: true # noqa ignore-errors
changed_when: false
Expand Down
2 changes: 1 addition & 1 deletion roles/etcd/tasks/join_etcd-events_member.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
etcd_events_peer_addresses: >-
{% for host in groups['etcd'] -%}
{%- if hostvars[host]['etcd_events_member_in_cluster'].rc == 0 -%}
{{ "etcd" + loop.index | string }}=https://{{ hostvars[host].etcd_events_access_address | default(hostvars[host].ip | default(hostvars[host]['fallback_ip'])) }}:2382,
{{ "etcd" + loop.index | string }}="https://{{ hostvars[host].etcd_events_access_address | default(hostvars[host]['main_ip']) | ansible.utils.ipwrap }}:2382",
{%- endif -%}
{%- if loop.last -%}
{{ etcd_member_name }}={{ etcd_events_peer_url }}
Expand Down
2 changes: 1 addition & 1 deletion roles/etcd/tasks/join_etcd_member.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
etcd_peer_addresses: >-
{% for host in groups['etcd'] -%}
{%- if hostvars[host]['etcd_member_in_cluster'].rc == 0 -%}
{{ "etcd" + loop.index | string }}=https://{{ hostvars[host].etcd_access_address | default(hostvars[host].ip | default(hostvars[host]['fallback_ip'])) }}:2380,
{{ "etcd" + loop.index | string }}="https://{{ hostvars[host].etcd_access_address | default(hostvars[host]['main_ip']) | ansible.utils.ipwrap }}:2380",
{%- endif -%}
{%- if loop.last -%}
{{ etcd_member_name }}={{ etcd_peer_url }}
Expand Down
4 changes: 2 additions & 2 deletions roles/etcd/templates/etcd-events.env.j2
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ ETCD_INITIAL_ADVERTISE_PEER_URLS={{ etcd_events_peer_url }}
ETCD_INITIAL_CLUSTER_STATE={% if etcd_events_cluster_is_healthy.rc == 0 | bool %}existing{% else %}new{% endif %}

ETCD_METRICS={{ etcd_metrics }}
ETCD_LISTEN_CLIENT_URLS=https://{{ etcd_address }}:2383,https://127.0.0.1:2383
ETCD_LISTEN_CLIENT_URLS=https://{{ etcd_address | ansible.utils.ipwrap }}:2383,https://127.0.0.1:2383
ETCD_ELECTION_TIMEOUT={{ etcd_election_timeout }}
ETCD_HEARTBEAT_INTERVAL={{ etcd_heartbeat_interval }}
ETCD_INITIAL_CLUSTER_TOKEN=k8s_events_etcd
ETCD_LISTEN_PEER_URLS=https://{{ etcd_address }}:2382
ETCD_LISTEN_PEER_URLS=https://{{ etcd_address | ansible.utils.ipwrap }}:2382
ETCD_NAME={{ etcd_member_name }}-events
ETCD_PROXY=off
ETCD_INITIAL_CLUSTER={{ etcd_events_peer_addresses }}
Expand Down
6 changes: 3 additions & 3 deletions roles/etcd/templates/etcd.env.j2
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ ETCD_METRICS={{ etcd_metrics }}
{% if etcd_listen_metrics_urls is defined %}
ETCD_LISTEN_METRICS_URLS={{ etcd_listen_metrics_urls }}
{% elif etcd_metrics_port is defined %}
ETCD_LISTEN_METRICS_URLS=http://{{ etcd_address }}:{{ etcd_metrics_port }},http://127.0.0.1:{{ etcd_metrics_port }}
ETCD_LISTEN_METRICS_URLS=http://{{ etcd_address | ansible.utils.ipwrap }}:{{ etcd_metrics_port }},http://127.0.0.1:{{ etcd_metrics_port }}
{% endif %}
ETCD_LISTEN_CLIENT_URLS=https://{{ etcd_address }}:2379,https://127.0.0.1:2379
ETCD_LISTEN_CLIENT_URLS=https://{{ etcd_address | ansible.utils.ipwrap }}:2379,https://127.0.0.1:2379
ETCD_ELECTION_TIMEOUT={{ etcd_election_timeout }}
ETCD_HEARTBEAT_INTERVAL={{ etcd_heartbeat_interval }}
ETCD_INITIAL_CLUSTER_TOKEN=k8s_etcd
ETCD_LISTEN_PEER_URLS=https://{{ etcd_address }}:2380
ETCD_LISTEN_PEER_URLS=https://{{ etcd_address | ansible.utils.ipwrap }}:2380
ETCD_NAME={{ etcd_member_name }}
ETCD_PROXY=off
ETCD_INITIAL_CLUSTER={{ etcd_peer_addresses }}
Expand Down
11 changes: 9 additions & 2 deletions roles/etcd/templates/openssl.conf.j2
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,16 @@ DNS.{{ counter["dns"] }} = {{ etcd_alt_name }}{{ increment(counter, 'dns') }}
{% if hostvars[host]['access_ip'] is defined %}
IP.{{ counter["ip"] }} = {{ hostvars[host]['access_ip'] }}{{ increment(counter, 'ip') }}
{% endif %}
IP.{{ counter["ip"] }} = {{ hostvars[host]['ip'] | default(hostvars[host]['fallback_ip']) }}{{ increment(counter, 'ip') }}
{% if hostvars[host]['access_ip6'] is defined %}
IP.{{ counter["ip"] }} = {{ hostvars[host]['access_ip6'] }}{{ increment(counter, 'ip') }}
{% endif %}
{% if ipv6_stack %}
IP.{{ counter["ip"] }} = {{ hostvars[host]['ip6'] | default(hostvars[host]['fallback_ip6']) }}{{ increment(counter, 'ip') }}
{% endif %}
IP.{{ counter["ip"] }} = {{ hostvars[host]['main_ip'] }}{{ increment(counter, 'ip') }}
{% endfor %}
{% for cert_alt_ip in etcd_cert_alt_ips %}
IP.{{ counter["ip"] }} = {{ cert_alt_ip }}{{ increment(counter, 'ip') }}
{% endfor %}
IP.{{ counter["ip"] }} = 127.0.0.1
IP.{{ counter["ip"] }} = 127.0.0.1{{ increment(counter, 'ip') }}
IP.{{ counter["ip"] }} = ::1
2 changes: 1 addition & 1 deletion roles/kubernetes/client/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@
user_certs: "{{ admin_kubeconfig['users'][0]['user'] }}"
username: "kubernetes-admin-{{ cluster_name }}"
context: "kubernetes-admin-{{ cluster_name }}@{{ cluster_name }}"
override_cluster_name: "{{ {'clusters': [{'cluster': (cluster_infos | combine({'server': 'https://' + external_apiserver_address + ':' + (external_apiserver_port | string)})), 'name': cluster_name}]} }}"
override_cluster_name: "{{ {'clusters': [{'cluster': (cluster_infos | combine({'server': 'https://' + (external_apiserver_address | ansible.utils.ipwrap) + ':' + (external_apiserver_port | string)})), 'name': cluster_name}]} }}"
override_context: "{{ {'contexts': [{'context': {'user': username, 'cluster': cluster_name}, 'name': context}], 'current-context': context} }}"
override_user: "{{ {'users': [{'name': username, 'user': user_certs}]} }}"
when: kubeconfig_localhost
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ kube_kubeadm_scheduler_extra_args: {}

# Associated interface must be reachable by the rest of the cluster, and by
# CLI/web clients.
kube_scheduler_bind_address: 0.0.0.0
kube_scheduler_bind_address: "::"

# ClientConnection options (e.g. Burst, QPS) except from kubeconfig.
kube_scheduler_client_conn_extra_opts: {}
Expand Down
6 changes: 3 additions & 3 deletions roles/kubernetes/control-plane/defaults/main/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ upgrade_cluster_setup: false
# listen on a specific address/interface.
# NOTE: If you specific address/interface and use loadbalancer_apiserver_localhost
# loadbalancer_apiserver_localhost (nginx/haproxy) will deploy on control plane nodes on 127.0.0.1:{{ loadbalancer_apiserver_port | default(kube_apiserver_port) }} too.
kube_apiserver_bind_address: 0.0.0.0
kube_apiserver_bind_address: "::"

# A port range to reserve for services with NodePort visibility.
# Inclusive at both ends of the range.
Expand All @@ -29,7 +29,7 @@ kube_etcd_key_file: node-{{ inventory_hostname }}-key.pem

# Associated interfaces must be reachable by the rest of the cluster, and by
# CLI/web clients.
kube_controller_manager_bind_address: 0.0.0.0
kube_controller_manager_bind_address: "::"

# Leader election lease durations and timeouts for controller-manager
kube_controller_manager_leader_elect_lease_duration: 15s
Expand Down Expand Up @@ -242,7 +242,7 @@ kubeadm_upgrade_auto_cert_renewal: true

## Enable distributed tracing for kube-apiserver
kube_apiserver_tracing: false
kube_apiserver_tracing_endpoint: 0.0.0.0:4317
kube_apiserver_tracing_endpoint: "[::]:4317"
kube_apiserver_tracing_sampling_rate_per_million: 100

# Enable kubeadm file discovery if anonymous access has been removed
Expand Down
4 changes: 2 additions & 2 deletions roles/kubernetes/control-plane/handlers/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@

- name: Control plane | wait for kube-scheduler
vars:
endpoint: "{{ kube_scheduler_bind_address if kube_scheduler_bind_address != '0.0.0.0' else 'localhost' }}"
endpoint: "{{ kube_scheduler_bind_address if kube_scheduler_bind_address != '::' else 'localhost' }}"
uri:
url: https://{{ endpoint }}:10259/healthz
validate_certs: false
Expand All @@ -92,7 +92,7 @@

- name: Control plane | wait for kube-controller-manager
vars:
endpoint: "{{ kube_controller_manager_bind_address if kube_controller_manager_bind_address != '0.0.0.0' else 'localhost' }}"
endpoint: "{{ kube_controller_manager_bind_address if kube_controller_manager_bind_address != '::' else 'localhost' }}"
uri:
url: https://{{ endpoint }}:10257/healthz
validate_certs: false
Expand Down
Loading

0 comments on commit a51e7dd

Please sign in to comment.