diff --git a/npep/npep-126-egress-traffic-control.md b/npep/npep-126-egress-traffic-control.md index 530df79b..2c62c5e6 100644 --- a/npep/npep-126-egress-traffic-control.md +++ b/npep/npep-126-egress-traffic-control.md @@ -1,7 +1,7 @@ # NPEP-126: Add northbound traffic support in (B)ANP API * Issue: [#126](https://github.com/kubernetes-sigs/network-policy-api/issues/126) -* Status: Provisional +* Status: Implementable ## TLDR @@ -76,13 +76,249 @@ selected cluster workloads to k8s-apiservers for securing the server. ## API -(... details, can point to PR with changes) +Proof of Concept for the API design details can be found here: https://github.com/kubernetes-sigs/network-policy-api/pull/143 +### Implementing egress traffic control towards cluster nodes + +This NPEP proposes to add a new type of `AdminNetworkPolicyEgressPeer` called `Nodes` +to be able to explicitly select nodes (based on the node's labels) in the cluster. +This ensures that if the list of IPs on a node OR list of nodes change, the users +don't need to manually intervene to include those new IPs. The label selectors will +take care of this automatically. + +``` +// AdminNetworkPolicyEgressPeer defines a peer to allow traffic to. +// Exactly one of the selector pointers must be set for a given peer. If a +// consumer observes none of its fields are set, they must assume an unknown +// option has been specified and fail closed. +// +kubebuilder:validation:MaxProperties=1 +// +kubebuilder:validation:MinProperties=1 +type AdminNetworkPolicyEgressPeer struct { + + // Nodes defines a way to select a set of nodes in + // in the cluster. This field follows standard label selector + // semantics; if present but empty, it selects all Nodes. + // + // Support: Core + // + // +optional + Nodes *metav1.LabelSelector `json:"nodes,omitempty"` +} +``` + +Note that `AdminNetworkPolicyPeer` will changed to +`AdminNetworkPolicyEgressPeer` and `AdminNetworkPolicyIngressPeer` since ingress and +egress peers have started to diverge at this point and it is easy to +maintain it with two sets of peer definitions. +This ensures nodes can be referred to only as "egress peers". + +Example: Admin wants to deny egress traffic from tenants who don't have +`restricted`, `confidential` or `internal` level security clearance +to control-plane nodes at 443 and 6443 ports in the cluster + +``` +apiVersion: policy.networking.k8s.io/v1alpha1 +kind: AdminNetworkPolicy +metadata: + name: node-as-egress-peers +spec: + priority: 55 + subject: + namespaces: + matchExpressions: + - {key: security, operator: notIn, values: [restricted, confidential, internal]} + egress: + - name: "deny-all-egress-to-kapi-server" + action: "Deny" + to: + - nodes: + matchLabels: + node-role.kubernetes.io/control-plane: true + ports: + - portNumber: + protocol: TCP + port: 443 + - portNumber: + protocol: TCP + port: 6443 +``` + +### Implementing egress traffic control towards external destinations + +This NPEP proposes to add a new type of `AdminNetworkPolicyEgressPeer` called `ExternalNetworks` +to be able to explicitly select external destinations (based on the `externalNetworkSet`'s +labels) in the cluster. This peer type will not be supported in `AdminNetworkPolicyIngressPeer`. + +``` +// AdminNetworkPolicyEgressPeer defines a peer to allow traffic to. +// Exactly one of the selector pointers must be set for a given peer. If a +// consumer observes none of its fields are set, they must assume an unknown +// option has been specified and fail closed. +// +kubebuilder:validation:MaxProperties=1 +// +kubebuilder:validation:MinProperties=1 +type AdminNetworkPolicyEgressPeer struct { + + // ExternalNetworks defines a way to select ExternalNetworkSets + // that consist of network CIDRs that live outside the cluster as a peer. + // This field follows standard label selector semantics; if present + // but empty, it selects all ExternalNetworkSets defined in the cluster. + // + // Support: Core + // + // +optional + ExternalNetworks *metav1.LabelSelector `json:"externalNetworks,omitempty"` +} +``` + +An `externalNetworkSet` is a new object used to define a set of networks outside +the cluster. This is defined as a new object so that its easy for users to group +their external entities and then refer to that group from the different egress rules. +This ensures consistency without the users having to manually change the ANP/BANP +to accommodate the changing needs of the network cidrs. They can directly edit the +new CRD which guarantees all rules effecting that peer will take effect automatically. +In the future when we define `DeveloperNetworkPolicies`, this will come even more handy +for uniformity specially across `Pass` actions. + +``` +// ExternalNetworkSet is a cluster level resource that is used to define +// a set of networks outsides the cluster which can be referred to from +// the AdminNetworkPolicy && BaselineAdminNetworkPolicy APIs as an external peer +type ExternalNetworkSet struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata"` + + // Specification of the desired behavior of ExternalNetworkSet. + Spec ExternalNetworkSetSpec `json:"spec"` +} + +// ExternalNetworkSetSpec defines the desired state of ExternalNetworkSet. +type ExternalNetworkSetSpec struct { + // Networks is the list of NetworkCIDR (both v4 & v6) that can be used to define + // external destinations. + // A total of 100 CIDRs will be allowed in each NetworkSet instance. + // ANP & BANP APIs may use the .spec.in(e)gress.from(to).externalNetworks selector + // to select a set of external networks + // +optional + // +kubebuilder:validation:MaxItems=100 + Networks []string `json:"networks,omitempty" validate:"omitempty,dive,cidr"` +} +``` + +Example: Let's define some sample sets of externalNetworks. + +``` +apiVersion: policy.networking.k8s.io/v1alpha1 +kind: ExternalNetworkSet +metadata: + name: api-vm-network + labels: + network: intranet +spec: + networks: + - 10.0.0.10/32 + - 10.0.0.11/32 + - 10.0.0.12/32 +--- +apiVersion: policy.networking.k8s.io/v1alpha1 +kind: ExternalNetworkSet +metadata: + name: storage-network + labels: + network: intranet +spec: + networks: + - 95.90.241.22/28 + - 200.0.65.0/24 +--- +apiVersion: policy.networking.k8s.io/v1alpha1 +kind: ExternalNetworkSet +metadata: + name: dns-network + labels: + network: dns +spec: + networks: + - 195.90.241.22/28 + - 111.0.65.0/29 +--- +apiVersion: policy.networking.k8s.io/v1alpha1 +kind: ExternalNetworkSet +metadata: + name: global-network + labels: + network: internet +spec: + networks: + - 0.0.0.0/0 +``` +``` +$ oc get ens +NAME NETWORKS AGE +api-vm-network ["10.0.0.10/32","10.0.0.11/32","10.0.0.12/32"] 2s +dns-network ["195.90.241.22/28","111.0.65.0/29"] 2s +global-network ["0.0.0.0/0"] 2s +storage-network ["95.90.241.22/28","200.0.65.0/24"] 2s +``` + +Let's define ANP and BANP that refer to these external networks: +``` +apiVersion: policy.networking.k8s.io/v1alpha1 +kind: AdminNetworkPolicy +metadata: + name: cluster-egress-controls +spec: + priority: 70 + subject: + namespaces: {} + egress: + - name: "allow-all-egress-to-intranet" + action: "Allow" + to: + - externalNetworks: + matchLabels: + network: intranet + - name: "allow-egress-to-external-dns" + action: "Allow" + to: + - externalNetworks: + matchLabels: + network: dns + ports: + - portNumber: + protocol: UDP + port: 53 + - name: "pass-egress-to-internet" + action: "Pass" + to: + - externalNetworks: + matchLabels: + network: internet +--- +apiVersion: policy.networking.k8s.io/v1alpha1 +kind: BaselineAdminNetworkPolicy +metadata: + name: default +spec: + subject: + namespaces: {} + egress: + - name: "deny-egress-to-internet" + action: "Deny" + to: + - externalNetworks: + matchLabels: + network: internet +``` +This allows admins to specify all cluster workloads can talk to +the company's internet CIDR's and the external DNS network. It +also allows them to put up default guardrails of not allowing +any other egress traffic towards internet in a BANP. ## Alternatives -(List other design alternatives and why we did not go in that -direction) +* Instead of defining a new object called `ExternalNetworkSet`, +we can go ahead with putting the CIDR ranges directly into the +main ANP/BANP objects. ## References