Feature Request: Get notification on CSR approval / denial

[wilreichert]

It'd be exceptionally useful to get a notification on CSR activity. Something like a webhook would be sufficient but direct support for apps like slack would be handy as well.

[jnummelin]

Initially I feel slightly hesitant to add such support in kubelet-rubber-stamper. It's not really part of it's scope to integrate to various apps/hooks. And I see many requests coming for different apps & hooks to integrate with.

I do however like the general idea of getting a notification in our Slack when new CSR is approved. 😄

I see there are some external generalized building blocks that could send notification from any event on kube api. For example: https://github.com/alauda/event-exporter

That would of course require kubelet-rubber-stamp to actually emit those events, but that should not be a big deal to make happen.

@wilreichert WDYT? Would running such an "exporter" in the cluster make sense for your use case?

[wilreichert]

I find leaving a wide trail of breadcrumbs behind me is generally advantageous. =) Specifically I'm interested in a) Slack and b) an audit trail via our monitoring platform (datadog). You make a valid point though - its totally possible to leverage other mechanisms to watch CSR events within the cluster. Where notifications would be most useful would be when kubelet-rubber-stamp is attempting to approve and failing for $REASON.

https://github.com/bitnami-labs/kubewatch seems pretty in line with what I'm looking for given the appropriate events are getting generated. Slack is so ubiquitous these days that adding direct support for it wouldn't be a bad thing either.

[waynr]

I have been thinking that it would be nice for kubelet-rubber-stamp to emit events rather than (or in addition to) log messages when it decides not to approve a CSR. Currently this is what I see:

2020/01/16 23:11:46 Reconciling CertificateSigningRequest /csr-4728
2020/01/16 23:11:46 x509 CN %q doesn't match CSR username %q system:node:csr-4728 system:serviceaccount:csr-4728:csr-4728-serviceaccount
2020/01/16 23:11:46 csr csr-4728 not recognized as kubelet serving csr, tried: [certificatesigningrequests]

The reason I want events in addition to just log messages is that my test cases currently rely on parsing pod logs to verify that kubelet-rubber-stamp does not approve a given CSR. This forces me to run kubelet-rubber-stamp as a normal pod on worker nodes rather than my preference which is to run it as a static pod on the master node.

Unless someone has already implemented it, I think I'll take a look at hacking this together in the near future. At that time I can include support for CSR approval events as well as anything else anyone thinks would be useful.

[jnummelin]

@waynr I haven't had the time to look at the events, so feel free to take a punch at it. I think good first step would be to have events on both the approval and rejection.