Skip to content

Commit

Permalink
Update bgp-policy.md
Browse files Browse the repository at this point in the history
Signed-off-by: Hongliang Liu <[email protected]>
  • Loading branch information
hongliangl committed Jul 22, 2024
1 parent 56a32cd commit 8ce56c4
Showing 1 changed file with 42 additions and 38 deletions.
80 changes: 42 additions & 38 deletions docs/bgp-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,20 @@
- [BGPPeers](#bgppeers)
- [Usage examples](#usage-examples)
- [Combined Advertisements of Service, Pod, and Egress IPs](#combined-advertisements-of-service-pod-and-egress-ips)
- [Advertise Egress IPs to external BGP peers more than one hop](#advertise-egress-ips-to-external-bgp-peers-more-than-one-hop)
- [Advertise Egress IPs to external BGP peers with more than one hops](#advertise-egress-ips-to-external-bgp-peers-with-more-than-one-hops)
- [Limitations](#limitations)
<!-- /toc -->

## What is BGPPolicy?

`BGPPolicy` is a CRD API that allows users to initialize BGP process on selected Kubernetes Nodes and advertise Service
IPs, Pod IPs and Egress IPs to remote BGP peers.
`BGPPolicy` is a custom resource that allows users to initialize BGP process on selected Kubernetes Nodes and advertise
Service IPs, Pod IPs, and Egress IPs to remote BGP peers, facilitating the integration of Kubernetes cluster with external
BGP-enabled network.

## Prerequisites

BGPPolicy was introduced in v2.1 as an alpha feature. A feature gate, `BGPPolicy` must be enabled on the antrea-agent in
the `antrea-config` ConfigMap for the feature to work, like the following:
BGPPolicy was introduced in Antrea v2.1 as an alpha feature. A feature gate, `BGPPolicy` must be enabled on antrea-agent
in the `antrea-config` ConfigMap for the feature to work, like the following:

```yaml
kind: ConfigMap
Expand All @@ -42,13 +43,11 @@ data:
## The BGPPolicy resource
A BGPPolicy in Kubernetes is a Custom Resource Definition (CRD) object. Like all CRD objects, you can POST a BGPPolicy
definition to the API server to create a new instance. This CRD allows users to initiate the BGP process on selected
Kubernetes Nodes and advertise various IPs (Service IPs, Pod IPs, and Egress IPs) to remote BGP peers, facilitating
the integration of Kubernetes cluster with external BGP-enabled network.
definition to the API server to create a new instance.
The following specification creates a BGPPolicy object. This policy starts the BGP process with ASN 64512, listening on
port 179, on Nodes labeled with `bgp=enabled`. It advertises ClusterIPs, LoadBalancerIPs, and ExternalIPs to a BGP peer
at IP 192.168.77.200, which has ASN 65001 and listens on port 179:
The following manifest creates a BGPPolicy object. It can start BGP process with ASN `64512`, listening on port `179`,
on Nodes labeled with `bgp=enabled`. The process will advertise ClusterIPs, LoadBalancerIPs, and ExternalIPs to a BGP
peer at IP address `192.168.77.200`, which has ASN `65001` and listens on port `179`:

```yaml
apiVersion: crd.antrea.io/v1alpha1
Expand All @@ -75,46 +74,55 @@ spec:

### NodeSelector

The `nodeSelector` field specifies which Kubernetes Nodes the BGPPolicy applies to based on the labels specified. If
`nodeSelector` is empty, no Node will be selected. The field is mandatory.
The `nodeSelector` field selects which Kubernetes Nodes the BGPPolicy applies to based on the Node labels. The field is
mandatory.

**Note**: If multiple BGPPolicy objects select a Node, the effective one will be selected randomly.
**Note**: If multiple BGPPolicy objects select the same Node, the effective BGPPolicy will be chosen randomly.

### LocalASN

The `localASN` field defines the Autonomous System Number (ASN) used by the local BGP process. The available private ASN
range is 64512-65535. The field is mandatory.
The `localASN` field defines the Autonomous System Number (ASN) that the local BGP process uses. The available private
ASN range is `64512-65535`. The field is mandatory.

### ListenPort

The `listenPort` field specifies the port on which the BGP process listens. The default value is 179.
The `listenPort` field specifies the port on which the BGP process listens. The default value is 179. The valid port
range is `1-65535`.

### Advertisements

The `advertisements` field configures which IPs are advertised to BGP peers.

- `service`: Specifies how to advertise Service IPs. The `ipTypes` field lists the types of Service IPs to be advertised,
including `ClusterIP`, `ExternalIP`, and `LoadBalancerIP`.
- `pod`: Specifies how to advertise Pod IPs. Currently, only the Node IPAM Pod CIDR can be advertised by setting `pod:{}`.
- `egress`: Specifies how to advertise Egress IPs. Currently, all Egress IPs can be advertised by setting `egress:{}`.
which can be `ClusterIP`, `ExternalIP`, and `LoadBalancerIP`.
- `pod`: Specifies how to advertise Pod IPs. The Node IPAM Pod CIDRs will be advertised by setting `pod:{}`.
- `egress`: Specifies how to advertise Egress IPs. All Egress IPs will be advertised by setting `egress:{}`.

### BGPPeers

The `bgpPeers` field lists the BGP peers to which the advertisements are sent.

- `address`: The IP address on which the BGP peer listens.
- `address`: The IP address of the BGP peer.
- `asn`: The Autonomous System Number of the BGP peer.
- `port`: The port number on which the BGP peer listens. Default is 179.
- `port`: The port number on which the BGP peer listens. The default value is 179.
- `multihopTTL`: The Time To Live (TTL) value used in BGP packets sent to the BGP peer, with a range of 1 to 255.
Default is 1.
The default value is 1.
- `gracefulRestartTimeSeconds`: Specifies how long the BGP peer waits for the BGP session to re-establish after a
restart before deleting stale routes, with a range of 1 to 3600 seconds. Default is 120 seconds.
restart before deleting stale routes, with a range of 1 to 3600 seconds. The default value is 120 seconds.

## BGP router ID

Only an IPv4 address can be used as the BGP Identifier (BGP router ID). For IPv4-only or dual-stack Kubernetes clusters,
the Node's IPv4 address is used as the BGP router ID, ensuring uniqueness. For IPv6-only Kubernetes clusters without a
Node IPv4 address, the router ID must be specified in the Node annotation `node.antrea.io/bgp-router-id`. If the
annotation is not present, an IPv4 address will be generated by hashing the Node name and updated to the annotation.
If the annotation is present, its value will be used as the BGP router ID.

## Usage examples

### Combined Advertisements of Service, Pod, and Egress IPs

In this example, we will advertise ClusterIP and LoadBalancerIP types of Service IPs, Pod CIDR, and Egress IPs from the
In this example, we will advertise ClusterIP and LoadBalancerIP types of Service IPs, Pod CIDRs, and Egress IPs from the
selected Nodes to multiple remote BGP peers.

```yaml
Expand Down Expand Up @@ -144,29 +152,24 @@ spec:
port: 179
```

### Advertise Egress IPs to external BGP peers more than one hop
### Advertise Egress IPs to external BGP peers with more than one hops

In this example, we configure the BGPPolicy to advertise Egress IPs from selected Nodes to a remote BGP peer that is
more than one hop away from the cluster. It's crucial to set the multihopTTL to a value greater than 1, which allows BGP
packets to traverse multiple hops to reach the peer.
In this example, we configure the BGPPolicy to advertise Egress IPs from selected Nodes to a remote BGP peer located
multiple hops away from the cluster. It's crucial to set the `multihopTTL` to a value equal to or greater than the
number of hops, allowing BGP packets to traverse multiple hops to reach the peer.

```yaml
apiVersion: crd.antrea.io/v1alpha1
kind: BGPPolicy
metadata:
name: advertise-all-ips
name: advertise-all-egress-ips
spec:
nodeSelector:
matchLabels:
bgp: enabled
localASN: 64512
listenPort: 179
advertisements:
service:
ipTypes:
- ClusterIP
- LoadBalancerIP
pod: {}
egress: {}
bgpPeers:
- address: 192.168.78.201
Expand All @@ -178,8 +181,9 @@ spec:
## Limitations

- The routes received from remote BGP peers will not be installed. Therefore, you must ensure that the path from Pods
to the remote BGP network is properly routable. This involves configuring your network infrastructure to handle the
routing of traffic between your Kubernetes cluster and the remote BGP network.
- IPv6 BGP peer is not supported in Windows.
to the remote BGP network is properly configured and routable. This involves configuring your network infrastructure
to handle the routing of traffic between your Kubernetes cluster and the remote BGP network.
- Only Linux Nodes are supported. The feature has not been validated on Windows Nodes, though theoretically it can work
with Windows Nodes.
- Advanced BGP features such as BGP communities, route filtering, route reflection, confederations, and other BGP policy
mechanisms defined in BGP RFCs are not supported.

0 comments on commit 8ce56c4

Please sign in to comment.