Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not assume systemd-resolved for resolv.conf #11813

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Do not assume systemd-resolved for resolv.conf #11813

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion roles/kubernetes/node/defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ kubelet_address: "{{ ip | default(fallback_ip) }}{{ (',' + ip6) if enable_dual_s
kubelet_bind_address: "{{ ip | default('0.0.0.0') }}"

# resolv.conf to base dns config
kube_resolv_conf: "/etc/resolv.conf"
kube_resolv_conf: "{{ '/run/systemd/resolve/resolv.conf' if 'systemd-resolved' in active_dns_services else '/etc/resolv.conf' }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that some resolv.conf modes (static if I remember correctly) check /etc/resolv.conf for files.
Using the systemd-resolved enablement as a check doesn't seem to be accurate.

kube_resolv_conf is by default still /etc/resolv.conf.
If systemd-resolved is enabled and /etc/resolv.conf is a soft link, change kube_resolv_conf to /run/systemd/resolve/resolv.conf.

What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/etc/resolv.conf is a soft link for /run/systemd/resolve/stub-resolv.conf not /run/systemd/resolve/resolv.conf .

And from the advice of https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues ,
So changing the config to run/systemd/resolve/resolv.conf is OK.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, what I'm trying to say is that systemd-resolved enablement is not the only way to tell, if systemd-resolved is enabled but /etc/resolv.conf is a file (not a soft link), it should probably be /etc/resolv.conf instead of /run/systemd/resolve/resolv.conf.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, if systemd-resolved is enabled/running but /etc/resolv.conf is not a symlink to either /run/systemd/resolve/stub-resolv.conf / /run/systemd/resolve/resolv.conf / /usr/lib/systemd/resolv.conf, systemd-resolved will use it as source of DNS configuration. See this excerpt from man systemd-resolved (8):

/ETC/RESOLV.CONF
       Four modes of handling /etc/resolv.conf (see resolv.conf(5)) are supported:

       •   systemd-resolved maintains the /run/systemd/resolve/stub-resolv.conf file for compatibility with traditional Linux programs. This file lists the
           127.0.0.53 DNS stub (see above) as the only DNS server. It also contains a list of search domains that are in use by systemd-resolved. The list of
           search domains is always kept up-to-date. Note that /run/systemd/resolve/stub-resolv.conf should not be used directly by applications, but only
           through a symlink from /etc/resolv.conf. This file may be symlinked from /etc/resolv.conf in order to connect all local clients that bypass local
           DNS APIs to systemd-resolved with correct search domains settings. This mode of operation is recommended.

       •   A static file /usr/lib/systemd/resolv.conf is provided that lists the 127.0.0.53 DNS stub (see above) as only DNS server. This file may be symlinked
           from /etc/resolv.conf in order to connect all local clients that bypass local DNS APIs to systemd-resolved. This file does not contain any search
           domains.

       •   systemd-resolved maintains the /run/systemd/resolve/resolv.conf file for compatibility with traditional Linux programs. This file may be symlinked
           from /etc/resolv.conf and is always kept up-to-date, containing information about all known DNS servers. Note the file format's limitations: it does
           not know a concept of per-interface DNS servers and hence only contains system-wide DNS server definitions. Note that
           /run/systemd/resolve/resolv.conf should not be used directly by applications, but only through a symlink from /etc/resolv.conf. If this mode of
           operation is used local clients that bypass any local DNS API will also bypass systemd-resolved and will talk directly to the known DNS servers.

       •   Alternatively, /etc/resolv.conf may be managed by other packages, in which case systemd-resolved will read it for DNS configuration data. In this
           mode of operation systemd-resolved is consumer rather than provider of this configuration file.

       Note that the selected mode of operation for this file is detected fully automatically, depending on whether /etc/resolv.conf is a symlink to
       /run/systemd/resolve/resolv.conf or lists 127.0.0.53 as DNS server.

So I think relying on systemd-resolved alone might actually work 🤔

I was initially going with checking for a symlink, but I'm not sure we should do that, because we end up with several boolean instead of ones, which results in ambiguity in certain cases:

  • /etc/resolv.conf is a symlink to one of the managed files, but systemd-resolved is not started/enabled. Or the reverse.
  • for that matter, systemd-resolved is started, but not enabled, or the reverse.

Wdyt ?


# Set to empty to avoid cgroup creation
kubelet_enforce_node_allocatable: "\"\""
Expand Down
2 changes: 0 additions & 2 deletions roles/kubernetes/node/vars/fedora.yml
View file
Open in desktop

This file was deleted.

2 changes: 0 additions & 2 deletions roles/kubernetes/node/vars/ubuntu-18.yml
View file
Open in desktop

This file was deleted.

2 changes: 0 additions & 2 deletions roles/kubernetes/node/vars/ubuntu-20.yml
View file
Open in desktop

This file was deleted.

2 changes: 0 additions & 2 deletions roles/kubernetes/node/vars/ubuntu-22.yml
View file
Open in desktop

This file was deleted.

2 changes: 0 additions & 2 deletions roles/kubernetes/node/vars/ubuntu-24.yml
View file
Open in desktop

This file was deleted.

26 changes: 11 additions & 15 deletions roles/kubernetes/preinstall/tasks/0020-set_facts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,21 +40,17 @@
src: /etc/resolv.conf
register: resolvconf_slurp

- name: NetworkManager | Check if host has NetworkManager
# noqa command-instead-of-module - Should we use service_facts for this?
command: systemctl is-active --quiet NetworkManager.service
register: networkmanager_enabled
failed_when: false
changed_when: false
check_mode: false

- name: Check systemd-resolved
# noqa command-instead-of-module - Should we use service_facts for this?
command: systemctl is-active systemd-resolved
register: systemd_resolved_enabled
failed_when: false
changed_when: false
check_mode: false
- name: Register which network services are active
systemd:
name: "{{ item }}.service"
loop:
- NetworkManager
- systemd-resolved
register: network_services

- name: Save list of active network service
set_fact:
active_dns_services: "{{ network_services.results | selectattr('status.ActiveState', '==', 'active') | map(attribute='item') }}"

- name: Set default dns if remove_default_searchdomains is false
set_fact:
Expand Down
2 changes: 1 addition & 1 deletion roles/kubernetes/preinstall/tasks/0063-networkmanager-dns.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
backup: true
when:
- ('127.0.0.53' not in nameserverentries
or systemd_resolved_enabled.rc != 0)
or 'systemd-resolved' not in active_dns_services)
notify: Preinstall | update resolvconf for networkmanager

- name: Set default dns if remove_default_searchdomains is false
Expand Down
33 changes: 16 additions & 17 deletions roles/kubernetes/preinstall/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@
when:
- dns_mode != 'none'
- resolvconf_mode == 'host_resolvconf'
- systemd_resolved_enabled.rc != 0
- networkmanager_enabled.rc != 0
- ('systemd-resolved' not in active_dns_services )
- ('NetworkManager' not in active_dns_services )
tags:
- bootstrap-os
- resolvconf
Expand All @@ -40,27 +40,26 @@
when:
- dns_mode != 'none'
- resolvconf_mode == 'host_resolvconf'
- systemd_resolved_enabled.rc == 0
- ('systemd-resolved' in active_dns_services )
tags:
- bootstrap-os
- resolvconf

- name: Apply networkmanager unmanaged devices settings
import_tasks: 0062-networkmanager-unmanaged-devices.yml
when:
- networkmanager_enabled.rc == 0
tags:
- bootstrap-os

- name: Apply networkmanager DNS settings
import_tasks: 0063-networkmanager-dns.yml
when:
- dns_mode != 'none'
- resolvconf_mode == 'host_resolvconf'
- networkmanager_enabled.rc == 0
- name: NetworkManager | Apply settings
when: ('NetworkManager' in active_dns_services )
tags:
- bootstrap-os
- resolvconf
block:
- name: Apply networkmanager unmanaged devices settings
import_tasks: 0062-networkmanager-unmanaged-devices.yml

- name: Apply networkmanager DNS settings
import_tasks: 0063-networkmanager-dns.yml
when:
- dns_mode != 'none'
- resolvconf_mode == 'host_resolvconf'
tags:
- resolvconf

- name: Install required system packages
import_tasks: 0070-system-packages.yml
Expand Down