[Filebeat] udp/tcp/netflow/lumberjack - support Proxy Protocol

[andrewkroh]

Describe the enhancement:

Support accepting ingress traffic that contains Proxy Protocol ¹ v1 or v2 headers.

Describe a specific use case for the enhancement or feature:

Allow users to put a load balancer (LB) in front of Filebeat and still have the original source address passed to the udp, tcp, netflow, and lumberjack inputs. Without this feature Filebeat will include the source address of the load balancer into the events which would accurately reflect the source of the data.

This is particularly useful for Netflow because the netflow and IPFIX RFCs state the collectors should use the source address as part of the association between templates and data records ². Without the real source address the netflow input cannot properly recall the appropriate template definitions for an exporter because the source port from the LB might be changing as it forwards packets.

As Template IDs are unique per UDP session and per Observation
Domain, at any given time, the Collecting Process SHOULD maintain the
following for all the current Template Records and Options Template
Records: <IPFIX Device, Exporter source UDP port, Collector IP
address, Collector destination UDP port, Observation Domain ID,
Template ID, Template Definition, Last Received>.

Load Balancers (not an exhaustive list)

• AWS Classic Load Balancer (TCP) (v1) - https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-proxy-protocol.html
• AWS Network Load Balancer (TCP/UDP) (v2) - https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#proxy-protocol
• GCP TCP Load Balancer (TCP) (v1) - https://cloud.google.com/sdk/gcloud/reference/compute/target-tcp-proxies/create
• Traefik (TCP/UDP) (v1, v2)- https://doc.traefik.io/traefik/routing/entrypoints/#proxyprotocol
• HAProxy (TCP) (v1, v2) - send-proxy and send-proxy-v2
• Nginx (TCP/UDP) (v2) - https://nginx.org/en/docs/stream/ngx_stream_proxy_module.html#proxy_protocol

Footnotes

1. https://www.haproxy.org/download/2.0/doc/proxy-protocol.txt ↩

2. https://www.rfc-editor.org/rfc/rfc7011#page-44 ↩

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[andrewkroh]

https://github.com/pires/go-proxyproto could be useful for implementation.

[elasticmachine]

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)