feat(pulsar sink): support tls options #22148
Conversation
|
|
github-actions
bot
added
domain: sources
| #[configurable_component] | ||
| #[configurable(description = "TLS options configuration for the Pulsar client.")] | ||
| #[derive(Clone, Debug)] | ||
| pub struct PulsarTlsOptions { |
There was a problem hiding this comment.
Would it be possible to reuse the existing config struct we use for TLS options:
?Or does the pulsar client no support all of the same options?
At the least, we should match the option names (e.g. certificate_chain_file should be ca_file to match).
There was a problem hiding this comment.
Thanks for your feedback.
rust client for pulsar only supports some options https://github.com/streamnative/pulsar-rs/blob/master/src/connection_manager.rs#L73
Changed to use the same field names as TlsConfig 66821d8.
There was a problem hiding this comment.
Nice work on this @pomacanthidae ! I appreciate you updating the integration tests including generating the test certificates. I left a question about the new config option.
pront
added
domain: security
| description: "TLS options configuration for the Pulsar client." | ||
| required: false | ||
| type: object: options: { | ||
| allow_insecure_connection: { |
There was a problem hiding this comment.
I think this should be called verify_certificate in order to match the existing options.
There was a problem hiding this comment.
Thanks, fixed in ed01093
There was a problem hiding this comment.
A couple of final comments, but otherwise 👍 Thanks for your work on this @pomacanthidae !
| @@ -0,0 +1,3 @@ | |||
| Tls options to set custom certificate chain are now available for `pulsar` sink and source. | |||
There was a problem hiding this comment.
| Tls options to set custom certificate chain are now available for `pulsar` sink and source. | |
| The `pulsar` source and sink now support configuration of TLS options via the `tls` configuration field. |
Cleaning this up a bit.
There was a problem hiding this comment.
fixed in 85de10c
src/sinks/pulsar/config.rs
Outdated
|
|
||
| #[configurable(derived)] | ||
| #[serde(default)] | ||
| pub(crate) tls_options: Option<PulsarTlsOptions>, |
There was a problem hiding this comment.
| pub(crate) tls_options: Option<PulsarTlsOptions>, | |
| pub(crate) tls: Option<PulsarTlsOptions>, |
Apologies, missed this one before. We should call this tls to match the other components.
There was a problem hiding this comment.
fixed in 59f0634
Head branch was pushed to by a user without write access
|
Oops, fixed the format to pass the test. |
Summary
This PR adds TLS options to enable Pulsar sinks and sources to use a custom certificate chain regarding #10888.
Integration tests for pulsar with TLS are added and test data in
tests/data/ca/intermediate_serverincluding pem files are generated by the following command.Change Type
Is this a breaking change?
How did you test this PR?
run integration test
Does this PR include user facing changes?
Checklist
Cargo.lock), pleaserun
dd-rust-license-tool writeto regenerate the license inventory and commit the changes (if any). More details here.References