From 0b2232b2e29dca5e2b6d9a5ca7f4b1ac394c9725 Mon Sep 17 00:00:00 2001 From: Abhishek Gaikwad Date: Tue, 26 Mar 2024 12:11:34 -0700 Subject: [PATCH] docs: external access and load balancer Signed-off-by: Abhishek Gaikwad --- docs/k8s_discussion.md | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/docs/k8s_discussion.md b/docs/k8s_discussion.md index 50433332..da47a66e 100644 --- a/docs/k8s_discussion.md +++ b/docs/k8s_discussion.md @@ -157,7 +157,7 @@ The choice between using in-cluster clients and external clients for AIStore has #### In-Cluster Clients: Simplified Management -- **Ease of Management**: In-cluster clients offer a more straightforward management experience. Their integration within the Kubernetes environment streamlines various processes. +- **Ease of Management**: In-cluster clients offer a more straightforward management experience. Their integration within the Kubernetes environment streamlines various processes. Daemons, including proxies and targets, are easily accessible via the `servicePort` specified in their configuration, facilitating inter-service communication. - **Utilizing Kubernetes DNS**: These clients can easily access AIStore endpoints using Kubernetes DNS names. This is particularly useful for connecting to the clusterIP proxy service, simplifying network configurations. @@ -165,26 +165,29 @@ The choice between using in-cluster clients and external clients for AIStore has #### External Clients: Additional Setup Required -- **Ingress Setup**: For external clients to access the AIStore cluster, you will need to establish ingress. This involves additional configuration steps not required for in-cluster clients. +- **Ingress Setup**: For external clients to access the AIStore cluster, you will need to establish ingress. This involves additional configuration steps not required for in-cluster clients. In the [deployment guide](README.md) we use `hostPort` to map a container's port to a corresponding port on the host machine to facilitate external access. -- **Port Configuration**: It's necessary to open specific ports for the targets and proxies to ensure external clients can connect. The necessary port information is detailed in the deployment guide. +- **Port Configuration**: It's necessary to open specific ports for the targets and proxies to ensure external clients can connect. The necessary port information is detailed in the [deployment guide](README.md). - **Performance Considerations**: Despite the differences in setup and management, the performance for in-cluster and external clients remains consistent. Both client types can achieve similar levels of efficiency and speed in data handling. -#### LoadBalancer and Ingress - -- **Load Balancer Requirement**: When using external clients, it's recommended to have a load balancer in place. This ensures clients can connect to a single, well-known IP address or DNS entry. +> Note: For deploying multiple targets on a single host machine, please refer our [documentation](multiple_targets_per_node.md). -- **Ingress Service on Proxy ClusterIP**: You'll need to set up a LoadBalancer type ingress service targeting the AIStore proxy's clusterIP service. The purpose here is not to perform actual load balancing (as Kubernetes proxy/IPVS will handle this) but to direct traffic to the AIStore proxies. - -- **High Availability Proxy Endpoint**: With several proxy Pods supporting the clusterIP service, this configuration effectively creates a highly available (HA) proxy endpoint. +#### LoadBalancer and Ingress -#### Specific Solutions for Different Environments +When using external clients, it's recommended to have a load balancer in place. This ensures clients can connect to a single, well-known IP address or DNS entry. To setup a load balancer you will need **external IPs**. The number of external IPs needed equals the number of targets plus one for the proxy. +**Setting up external IPs** - **Bare-Metal On-Premises Deployments**: For these setups, we recommend using [MetalLB](https://metallb.universe.tf/), a popular solution for on-premises Kubernetes environments. - - **Cloud-Based Deployments**: If your AIStore is running in a cloud environment, you can utilize standard HTTP load balancer services provided by the cloud provider. +- **Proxy and Target Load Balancers**: + - **Proxy LB**: A single load balancer consolidates proxy access, creating a high-availability endpoint for the clusterIP service. + - **Target LBs**: Individual load balancers for each target direct traffic to specific AIStore targets, facilitating ingress rather than distributing load. + +**Automating Load Balancer Setup**: +You can manually configure your load balancers or enable automatic setup by setting `externalLB` to `true` in your AIStore Custom Resource specification, allowing the AIS Operator to handle the configuration on your behalf. + ### Host Performance Tuning To efficiently handle high HTTP GET/PUT loads in AIStore, several tuning parameters are necessary, focusing on socket counts, port numbers, and port reuse. These are detailed in the provided [playbooks](../playbooks). For setups with high-performance expectations or high bandwidth networking, additional tuning, especially for HDDs involving I/O scheduler adjustments, is recommended and also outlined in the playbooks. For further guidance on enhancing AIStore's performance, refer to the supplementary [document](https://aiatscale.org/docs/performance).