Enable QDR port 5671 to listen on both IPv4 and IPv6 #599
Conversation
|
Does the CI have IPV6 only env test case?? |
|
It looks like the change I made has impact on SG's for IPV6 only clusters. Can someone provide insights on which component may be causing this problem, I am guessing that somewhere in SG also it is configured to connect for IPV4 only of AMQP |
No |
The SG connects to AMQP from the sg-bridge component, which needs a small fix here: infrawatch/sg-bridge#37 but this seems like a different problem. Since it says cookie.go, it must be from sg-core (it's written in golang, but the bridge is in C). I'm guessing it's probably in the prometheus exporter code, but I can't seem to put my finger on it yet. Can you provide any additional logging context or stack trace for that error? |
Logging context: 2024/05/24 13:42:57 provider.go:129: Defaulting client-id to system:serviceaccount:service-telemetry:smart-gateway |
Thanks, this makes more sense, now! The error is coming from the oauth-proxy container, not from the SG itself. It looks like this may be a problem in golang's cookie handling, though the ultimate impact is unclear without some additional testing. Here is another report of non-ipv6 compatible behavior in golang's cookie.go golang/go#65521 |
| @@ -227,7 +227,7 @@ | |||
| prefix: ceilometer | |||
| edgeListeners: | |||
| - expose: true | |||
| host: 0.0.0.0 | |||
| host: '' | |||
There was a problem hiding this comment.
Just note:
While documentation [1] says that empty string makes QDR listen on all local interfaces and [::] makes QDR listen on all IPv6 addresses, my manual testing with those two values was not successful:
case 1:
2024-05-29 19:04:10.899935 +0000 CONN_MGR (info) Configured Listener: '':5999 proto=any, role=normal
2024-05-29 19:04:10.909703 +0000 SERVER (notice) Operational, 4 Threads Running (process ID 1)
2024-05-29 19:04:10.909981 +0000 SERVER (notice) Process VmSize 114.67 MiB (125.40 GiB available memory)
2024-05-29 19:04:10.910211 +0000 SERVER (error) Listener error on '':5999: Name or service not known - listen on '':5999 (proton:io)
2024-05-29 19:04:10.910241 +0000 SERVER (critical) Shutting down, required listener failed '':5999
case2:
2024-05-29 19:11:11.596032 +0000 CONN_MGR (info) Configured Listener: [::]:5999 proto=any, role=normal
2024-05-29 19:11:11.601813 +0000 SERVER (notice) Operational, 4 Threads Running (process ID 1)
2024-05-29 19:11:11.601937 +0000 SERVER (notice) Process VmSize 114.67 MiB (125.40 GiB available memory)
2024-05-29 19:11:11.602230 +0000 SERVER (error) Listener error on [::]:5999: Name or service not known - listen on [::]:5999 (proton:io)
2024-05-29 19:11:11.602280 +0000 SERVER (critical) Shutting down, required listener failed [::]:5999
The only possible way to force QDR to listen on IPv6 address was to use the address explicitly.
[1] https://qpid.apache.org/releases/qpid-dispatch-1.17.0/man/qdrouterd.conf.html#_listener
There was a problem hiding this comment.
This test is not official since my local build environment was giving me trouble today, but:
- I edited my Interconnect object to say
host: ''like in this patch - OCP returns it to me with empty double-quotes instead
$ oc get Interconnect -o yaml | grep host
host: ""
- The operator renders the config file with no host field at all
$ oc rsh pod/default-interconnect-574d468757-wshtn grep -C 4 "port: 5671" /opt/interconnect/etc/qdrouterd.conf
}
listener {
role: edge
port: 5671
saslMechanisms: PLAIN
authenticatePeer: true
sslProfile: openstack
}
- qdr is listening on both protocols
$ oc rsh pod/default-interconnect-574d468757-wshtn netstat -anp | grep :5671
tcp 0 0 0.0.0.0:5671 0.0.0.0:* LISTEN 1/qdrouterd
tcp6 0 0 :::5671 :::* LISTEN 1/qdrouterd
Note that this was done on top of upstream builds and by manually editing the Interconnect object, not from actually running this PR code. I'll need to figure out what's wrong with my cluster; I think I'm just using it for too many things at once.
There was a problem hiding this comment.
I also tried this with no host field in the Interconnect object at all, and the results were the same. Maybe that's the preferred way to write it to prevent any future doubts about the problems Martin has shown.
| @@ -227,7 +227,7 @@ | |||
| prefix: ceilometer | |||
| edgeListeners: | |||
| - expose: true | |||
| host: 0.0.0.0 | |||
| host: '' | |||
There was a problem hiding this comment.
| host: '' |
|
This change depends on a change that failed to merge. Change infrawatch/sg-bridge#37 is needed. |
Co-authored-by: Chris Sibbitt <[email protected]>
|
Can we please get an insight why infrawatch/sg-bridge#37 does not pass in CI, as these two both are needed for IPv6 to work (plus potentially more?) |
|
I am updating this PR to synch with master and include #603, so that it doesn't get stuck in the gate. |
|
Build failed (check pipeline). Post https://review.rdoproject.org/zuul/buildset/3774ffdb897742a4943a19c165568454 ✔️ stf-crc-ocp_412-local_build SUCCESS in 37m 33s |
|
recheck |
|
Build failed (check pipeline). Post https://review.rdoproject.org/zuul/buildset/531a6b4354344aaf9c2972c45fb94388 ❌ stf-crc-ocp_412-local_build FAILURE in 24m 22s |
|
recheck |
|
recheck |
|
Build failed (check pipeline). Post https://review.rdoproject.org/zuul/buildset/1fcf852d1d8a4bbf89d3917a4a1d8eb6 ❌ stf-crc-ocp_412-local_build FAILURE in 28m 43s |
|
The zuul jobs are blocked by this change: infrawatch/sg-core#135, which will shortly be resolved |
Summary
This PR updates the Qpid Dispatch Router (QDR) configuration to allow port 5671 to listen on all local addresses, both IPv4 and IPv6. Previously, the port was configured to listen only on IPv4 addresses. With this change, QDR can now support single stack IPv6 clusters as well as dual stack configurations.
Changes
component_qdr.ymlto configure QDR to listen on all interfaces (both IPv4 and IPv6) on port 5671.Testing
Motivation and Context
This change is necessary to ensure compatibility with IPv6-only Kubernetes clusters, which are becoming more common as organizations prepare for the future of networking. Supporting both IPv4 and IPv6 ensures broader compatibility and future-proofs the service telemetry operator.
Related Issues