Skip to content

Latest commit

 

History

History
54 lines (37 loc) · 4.31 KB

kubernetes_security_architecture_considerations.md

File metadata and controls

54 lines (37 loc) · 4.31 KB
Raw

Kubernetes Security Architecture Considerations

This is an (at the moment) random list of things to think about when architecting Kubernetes based systems. They may not all still be current and if you know one's not right, PRs always welcome :)

CVEs

Authentication

  • None of the built-in authentication mechanisms shipped with base k8s are suitable for use by users.
    • Token authentication requires clear text tokens on disk and an API server restart to change.
    • Client certificate authentication does not support revocation (Github issue here)
  • Kubernetes does not have a user database, it relies on identity information passed from any approved authentication mechanism.
    • This means that if you have multiple valid authentication mechanisms, there is a risk of duplicate user identities. N.B. Kubernetes audit logging does record the identity of the user, but not the authentication source.

RBAC

  • There are various RBAC rights that can allow for Privilege escalation (there's a Kubernetes Docs page on this with more information here)
    • GET or LIST on secrets at a cluster level (or possibly at a namespace level) will allow for privesc via service account secrets. N.B. LIST on its own will do this.
    • Access to ESCALATE, IMPERSONATE or BIND as RBAC verbs can allow privilege escalation.
  • The system:masters group is hard-coded into the API server and provides cluster-admin rights to the cluster.
    • Access by a user using this group bypasses authorization webhooks (i.e. the request is never sent to the webhook)
  • Access to the node/proxy resource allows for privilege escalation via the kubelet API. Users with this right can either go via the Kubernetes API server to access the kubelet API, or go directly to the kubelet API. The kubelet API does not have audit logs and its use bypasses admission control.

Networking

  • Pods allowed to use host networking bypass Kubernetes Network Policies.
  • Services allows to used nodePorts can't be limited by Kubernetes Network Policies.
  • The pod proxy feature can be used to access arbitrary IP addresses via the Kubernetes API server (i.e. connections come from the API server address), which may bypass network restrictions (more details here)

Pod Security Standards

  • Without restriction on privileged containers, pods can be used to escalate privileges to node access
  • Some capabilities like CAP_SYS_ADMINwill similarly allow for node access

Distributions

  • Many Kubernetes distributions provide a first user which uses a client certificate (so no revocation) bound to the system:masters group, so guaranteed cluster-admin.
  • Some distributions place sensitive information such as the API server certificate authority private key in a configmap (e.g. RKE1)

DNS

  • At the moment it is possible to use DNS to enumerate all pods and services in a cluster, which can make leak information especially in multi-tenant clusters. (CoreDNS Issue here) (script to demonstrate here)

Auditing

  • Kubernetes auditing is not enabled by default.
  • Allowing direct access to the kubelet API effectively bypasses auditing, so care should be taken in allowing this.
  • Whilst audit logging provides the user who made the request it doesn't log the authentication mechanism used. As such if there are multiple configured authentication mechanisms (e.g. certificate authentication and OIDC) there is a risk that an attacker can create a user account which would appear to be that of another legitimate user