dnsdist: Add a new, optional, YAML-based configuration format #14969
Conversation
Pull Request Test Coverage Report for Build 12356327481Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
@pieterlexis just suggested two ways of supporting Lua functions that I like very much, I'll look into it: - name: "Lua Action"
selector:
type: "QName"
qname: "no-backend.doh.tests.powerdns.com."
action:
type: "Lua"
lua: |
function dohHandler(dq)
if dq:getHTTPScheme() == 'https' and dq:getHTTPHost() == '%s:%d' and dq:getHTTPPath() == '/' and dq:getHTTPQueryString() == '' then
local foundct = false
for key,value in pairs(dq:getHTTPHeaders()) do
if key == 'content-type' and value == 'application/dns-message' then
foundct = true
break
end
end
if foundct then
dq:setHTTPResponse(200, 'It works!', 'text/plain')
dq.dh:setQR(true)
return DNSAction.HeaderModify
end
end
return DNSAction.None
end - name: "Lua Action"
selector:
type: "QName"
qname: "no-backend.doh.tests.powerdns.com."
action:
type: "Lua"
lua-file: "/etc/dnsdist/my-action.lua" |
rgacogne
force-pushed
the
ddist-yaml-configuration-harder
branch
2 times, most recently
from
53a5493 to
e83c9b1
Compare
|
I cleaned up the commits history so it should be a lot easier to review this PR (which is still a big one, sorry about that). |
rgacogne
force-pushed
the
ddist-yaml-configuration-harder
branch
2 times, most recently
from
970895f to
94bb3ba
Compare
| }); | ||
| } | ||
|
|
||
| static void loadDynamicBlockConfiguration(const dnsdist::rust::settings::DynamicRulesSettingsConfiguration& settings, const ::rust::Vec<dnsdist::rust::settings::DynamicRulesConfiguration>& dynamicRules) |
Check warning
Code scanning / CodeQL
Poorly documented large function Warning
I pushed a commit implementing both the "inline" and the "detached to a file" options. Note that the final naming is slightly different. |
|
First very early impressions with some random remarks, only looking at a simple config I had laying around, trying to convert it. I did not look at code yet. See comments in yaml below # setStructuredLogging(true, {timeFormat="ISO8601"}) # no equivalent yaml found
console:
listen-address: 127.0.0.1:5300
key: "NtQDMjkGRTIBHJvFsSy/f33C50suAZQRwYWjOznnP+Q="
binds:
- listen-address: '127.0.0.1:5302'
protocol: DoH # allowed values absent in docs
doh:
paths: ['/dns-query'] # Seems required, opposed to what the docs say
tls:
certificates:
- certificate: fullchain.pem
key: mini.drijf.net.key
session-timeout: 30
backends:
- address: '9.9.9.9'
protocol: Do53 # allowed values absent in docs
health-checks:
mode: lazy # Is this the way? allowed values absent in docs
general:
verbose: true
verbose-health-checks: true # I saw no effect
load-balancing-policies:
default-policy: roundrobin # allowed values absent in docs``` |
|
When I did recursor, I went for underscores in names, instead of hyphens. I was under the impression that most YAML formats use underscores (at least Ansible does)... I wonder if we want to keep this consistent between products. If so, I'm afraid the burden is for you. |
I really hate underscores in names. I'll do it if I have to, but that sucks. |
Oh, I wont insist... there are many existing YAML formats using hyphens |
|
|
No, I think you are right. We need to be consistent across our products. I'm just disappointed because I have never been able to get underscores right and the current naming felt so natural to me, but it is what it is. |
Yeah, the rec ship has sailed...wondering if we migh be able to allow both styles in rec using Update: it seems possible with very little effort. Maybe something for the upcoming release cycle. |
I went back and forth quite a bit with these. In the end I think it makes sense to have them in git, and thus to remove them from the |
I'm ok with that. I seem to remember meson has some strict ideas about generating dist tarballs, requiring everything to be in git? |
Yep. I think we have other issues to handle to be able to handle |
rgacogne
force-pushed
the
ddist-yaml-configuration-harder
branch
from
89d6241 to
74f878e
Compare
|
Quite a few dangling refs in the docs now: |
| }); | ||
| } | ||
|
|
||
| if (!globalConfig.general.capabilities_to_retain.empty()) { |
There was a problem hiding this comment.
I already mentioned it elseweher, most general settingss do not seem to be implemented. I only found capabilities_to_retain here.
There was a problem hiding this comment.
Have you seen dnsdist-rust-lib/dnsdist-configuration-yaml-items-generated.cc?
There was a problem hiding this comment.
I'm sure I missed a few settings, to be clear, but I sure hope we are not missing most of them :)
There was a problem hiding this comment.
Oh, I think I only grepped in the cwd... sorry
Looking, thanks! I can no longer generate the documentation on Arch so I'm doing the generation on a remote box, and I think my script is not showing me the warnings. I'll fix it. |
To be consistent with what the recursor is already doing.
As suggested by Otto (thanks!)
There was a problem hiding this comment.
That's really nice, nothing caught my eye, lgtm.
At some point that would be good to port the dnsdistconf.lua file to yml (either keep both or replace it).
Building works fine on a mac except a few warnings https://gist.github.com/chbruyand/15f33ca27e300e9e5a34715fcb0b9e12
rgacogne
force-pushed
the
ddist-yaml-configuration-harder
branch
from
92c8679 to
01868f1
Compare
|
Thanks, guys! I pushed a few commits that should address most, if not all, of your comments. I still need to fix a missing descriptions in the documentation but I don't expect any big change in the code for this PR. |
|
Still quite a few dangling doc refs (happy to get those fixed after merge): |
I'm looking into it right now! |
Short description
This PR adds support for a new, optional, YAML-based configuration format. At this point this is still a work in progress!
If the configuration file passed to dnsdist via the
-Ccommand-line switch ends in.yml, it is assumed to be in the new YAML format, and an attempt to load a Lua configuration file with the same name but the.luawill be done before loading the YAML configuration. If the names ends in.lua, there will also be an attempt to find a file with the same name but ending in.yml. Otherwise the existing Lua configuration format is assumed.The parsing of the YAML configuration is done using Serde, so Rust is a requirement for the new configuration format, but the existing Lua configuration format is still supported so DNSdist can still be built without Rust. This might change in the future.
A fair amount of the new code is generated, based on a (YAML) description of the configuration settings, including the selectors and actions composing DNSdist's rules. Lua bindings for a fair amount of supported selectors and actions are now generated as well. My plan is to generate Lua bindings for most of the settings as well in the future.
The documentation of the new configuration format is not yet done, although a fair amount can already be generated from the definitions of settings, selectors and actions.
I have added several regression tests using the new configuration format, so for now it's possible to look at these to get a fell of the new format.
Checklist
I have: