NetBox DNS Use Cases

[peteeckel]

I'm starting to move over selected contents from the project's old home at auroraresearchlab/netbox-dns here in case the old repository should go away.

This should not deter anyone from providing their own use cases and examples. Please share what you think could be useful to other members of the community!

[peteeckel]

DNS provisioning with Ansible and NetBox DNS

Let me try to describe the boilerplate architecture I'm currently using for DNS automation with NetBox DNS. I think it's fairly common, but maybe it's not so obvious to other users approaching the issue.

DNS Architecture

The architecture of the DNS environments I maintain is usually more or less the same:

• One hidden master server with the zone data, running BIND
• One or more clusters of secondaries serving as authoritative name servers, running BIND as well
• A varying number of resolvers dispersed across network zones running unbound.

The hidden master is the only instance dealing with zone files, while the slave servers need the zone information for the configuration files. The resolvers only get the addresses of the slave servers to forward requests to.

In some cases the BIND servers are configured to use different zones for usually two views in a split horizon configuration, mostly to differentiate between the zones served to nodes in the internal network and the zones served to the Internet.

Automation

Automation is done using Ansible. This means both the automation of the name server and resolver setup as such and the automation of the provisioning of the data served by the DNS infrastructure.

The code listed below is provided as an example. It is not complete, does not contain any site-specific information and will not apply to a different Linux distribution, a different infrastructure, or generally any setup it was not created to work with. It contains, however, the most relevant parts needed to create a solution that will.

Infrastructure Setup Automation

The setup of the infrastructure itself (hidden master, slaves and resolvers) is currently not driven by NetBox. This is the point where the discussion started @LuPo in #248 kicks in, as some parts of the infrastructure such as IP ranges, TSIG authentication etc. associated with DNS, but in varying degrees depending on the tools and the architecture used, are not currently something NetBox DNS deals with.

Currently, the infrastructure is maintained in some YAML files that Ansible uses when provisioning. An example of such a configuration file is this:

named:
    views:
        internal:
            tsig_key: slave_internal
            masters:
                hosts:
                  - hiddenmaster
                tsig_keys:
                  - master_internal
            updaters:
                tsig_keys:
                  - updater_internal
            resolvers:
                tsig_keys:
                  - resolver_internal
            slaves:
                tsig_keys:
                  - slave_internal

        external:
            tsig_key: slave_external
            masters:
                hosts:
                  - hiddenmaster
                tsig_keys:
                  - master_external
            slaves:
                tsig_keys:
                  - slave_internet
                  - slave_external
                hosts:
                  - externalslave1
                addresses:
                  - fe80::dead:beef:1:1
                  - fe80::dead:beef:1:2
                  - fe80::dead:beef:2:1
                  - fe80::dead:beef:2:2
            updaters:
                tsig_keys:
                  - updater_external
            resolvers:
                tsig_keys:
                  - resolver_external
                acls:
                  - any

Here internal name servers are authenticated to accessing the views they are allowed to see via TSIG keys, while external slave servers and resolvers are identified by IP addresses or ACLs.

DNS Provisioning Automation

This part is where the data maintained in NetBox DNS come in. While the infrastructure part is largely static once infrastructure has been set up, the DNS data are highly dynamic and can change fairly often. There are two separate cases that have to be handled:

DNS Zone Changes

Changes to DNS Zones (such as the creation or deletion of zones) need to be applied to both the hidden master and the slaves serving views within the affected zones, and in some cases (though that is depending on how much information about the zones the resolvers need) to the resolvers.

Confiration files such as named.conf or unbound.conf, are compiled from the infrastructure information maintained in the YAML files, and the DNS zone and view data maintained in NetBox DNS. This is done by Ansibles ansible.builtin.template module and templates for BIND name servers or unbound resolvers. An example for a named.conf template (in this case, for a slave):

{% import 'macros.j2' as macros with context -%}

{{- macros.named_conf_common() -}}
{{- macros.used_acls() -}}
{{- macros.used_tsig_keys() -}}


{% for _view in named.views|dict2items() %}
{% if _view.key in nb_views|map(attribute='name') %}
{% set _view_config = _view.value %}
{% set _view_data = views[_view.key] -%}


view "{{ _view.key }}" {
{{- macros.view_permissions(_view_config) -}}
{{- macros.view_server_keys(_view_config) -}}
{{- macros.view_outgoing_keys(_view_config) -}}
{{- macros.view_forwarders(_view_config) -}}
{{- macros.view_common() -}}
{% else %}
# WARNING: View {{ _view.key }} is not defined
{% endif %}


{%- for _zone in nb_zones %}
{% if _zone.view.name == _view.key %}
{% if _zone.custom_fields.dynamic_zone|default(False) and _view_config.updaters|default(None) %}
{{- macros.zone(type='slave', view=_view.key, zone=_zone.name, masters=_view_config.masters.hosts, updaters=_view_config.updaters) -}}
{% else %}
{{- macros.zone(type='slave', view=_view.key, zone=_zone.name, masters=_view_config.masters.hosts) -}}
{% endif %}
{% if not loop.last %}

{% endif %}
{% endif %}
{% endfor %}
};
{% if not loop.last %}

{% endif %}
{% endfor %}

The macros include some boilerplate and the required TSIG keys and ACLs, which are generated from the YAML inventory file, a secure data vault and fixed text blocks.

The part that is created from data in NetBox DNS is contained in two variables populated when the Ansible playbook runs: nb_views and nb_zones:

- name: Get the view and zone lists from Netbox DNS
  ansible.builtin.set_fact:
      nb_views: "{{ query('netbox.netbox.nb_lookup', 'views', plugin='netbox_dns',
                          api_endpoint=netbox_api_endpoint,
                          validate_certs=netbox_api_validate,
                          token=netbox_api_token)
                    | map(attribute='value') }}"
      nb_zones: "{{ query('netbox.netbox.nb_lookup', 'zones', plugin='netbox_dns',
                          api_endpoint=netbox_api_endpoint,
                          api_filter='active=true, view__notequal=None',
                          validate_certs=netbox_api_validate,
                          token=netbox_api_token)
                    | map(attribute='value')
                    | rejectattr('view', 'equalto', None)
                    | rejectattr('nameservers', 'equalto', []) }}"
  ignore_errors: true
  register: netbox_api
  tags:
    - update_zones
    - update_records

This is where the Ansible lookup netbox.netbox.nb_lookup is used to pull the relevant data from NetBox DNS.

Based on these building blocks, the hidden master, the slaves and the resolvers are configured for all views and zones relevant to them.

Note that this part of the configuration is only required when there are changes to the zone structure itself, not when only records have been changed.

DNS Record Changes

Changes to DNS records are the much more common and, fortunately, much simpler case. Essentially, Ansible needs to create zone files on the hidden master and tell it to reload them, the rest is done by the DNS infrastructure itself.

In addition to the information from the nb_zones and nb_views variables mentioned above, now also the data from the Records model come into play. For each zone (or, from the perspective of BIND, each (view, zone)-tuple) the set of records needs to be determined and a zone file created from this set.

- name: Create the zone files for view "{{ view.name }}"
  ansible.builtin.template:
      src: zone.db.j2
      dest: "{{ named_data }}/{{ view.name }}/{{ dns_zone.name }}.db"
      validate: /sbin/named-checkzone {{ dns_zone.name }} %s
      force: "{{ not dns_zone.custom_fields.dynamic_zone|default(False) }}"
  loop: "{{ nb_zones|selectattr('view.name', 'equalto', view.name) }}"
  loop_control:
      label: "{{ dns_zone.name }}"
      loop_var: dns_zone
  vars:
      zone_records: "{{ query('netbox.netbox.nb_lookup', 'records', plugin='netbox_dns',
                              api_endpoint=netbox_api_endpoint,
                              api_filter='active=true, zone_id='+dns_zone.id|string,
                              validate_certs=netbox_api_validate,
                              token=netbox_api_token)
                       |map(attribute='value') }}"
  notify: reload named
  tags:
    - update_records
    - update_zones

This is all it needs to create a new zone file from this template:

;
; Zone file for zone {{ dns_zone.name }} [{{ dns_zone.view.name }}]
;

$TTL {{ dns_zone.default_ttl }}

{% for _record in zone_records|selectattr('status', 'equalto', 'active') %}
{% if _record.type in ['TXT', 'HINFO'] %}
{{ _record.name.ljust(32) }}    {{ (_record.ttl|string if _record.ttl is not none else '').ljust(8) }} IN {{ _record.type.ljust(8) }}    "{{ _record.value }}"
{% else %}
{{ _record.name.ljust(32) }}    {{ (_record.ttl|string if _record.ttl is not none else '').ljust(8) }} IN {{ _record.type.ljust(8) }}    {{ _record.value }}
{% endif %}
{% endfor %}

Enhancements

In many cases it would make sense to move configuration from the YAML files mentioned above to NetBox DNS or another NetBox module. Generally, information that is generic for DNS could be included in NetBox DNS, while information specific to a particular name server implementation would make more sense in a module specific to that name server.

One example of the former are TSIG keys, or the information whether a name server is a (hidden) master, a slave or a resolver. TSIG keys, however, are confidential information and should be stored in NetBox secrets (now a separate module), and access configuration to name servers based on TSIG keys, ACLs or addresses is in most cases depending on the implementation and not so generic to the DNS protocol itself.

Even views, already implemented in NetBox DNS, are not a standard DNS feature by themselves. They need to be understood as namespaces, and they are so specific to name servers that some implementations (e.g. PowerDNS) don't even have them. There was some discussion whether to call them "views" at all, because that relates to BIND, but nobody came up with a better name. On the other hand, they are a mandatory feature to make it possible for NetBox DNS to keep several zones with the same name, which is a pretty frequent use case.

My personal idea for this is to create some additional modules (maybe netbox-bind, netbox-powerdns etc.) that can be used to maintain the configuration that is depending on the name server implementation. Secrets should be stored in the netbox-secretstore module.

Then we need to define some enhancements to the NetBox DNS models that make it possible to link that information to DNS zones and views. Such an enhancement would make it possible to model the relations between name servers and resolvers within NetBox DNS.

Further Information

For some more information how to set up NetBox to work with Ansible, please see the code example provided with the plugin.