Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Documentation for Replication Using a Load Balancer #2375

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add Documentation for Replication Using a Load Balancer #2375

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions docs/nginx-ingress.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,3 +131,58 @@ spec:
port:
number: 9443
```

### Replication configuration:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not necessarly related to replication, this is mainly a very slow deployment or best practice were not followed, like a user storing a lot of objects under a single prefix

the one here works most of the time,

https://min.io/docs/minio/linux/integrations/setup-nginx-proxy-with-minio.html


Sometimes, you need to configure timeout in order to avoid Load Balancer to kill the connection while replication is taking place, for that, we can configure a read/write timeout of X minutes in the provided load balancer, you can add the appropriate NGINX annotations to your ingress configuration under `metadata.annotations`. The relevant annotations are:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rec based on Anis comment:

Suggested change
Sometimes, you need to configure timeout in order to avoid Load Balancer to kill the connection while replication is taking place, for that, we can configure a read/write timeout of X minutes in the provided load balancer, you can add the appropriate NGINX annotations to your ingress configuration under `metadata.annotations`. The relevant annotations are:
Ingress by default may time out connections before an operation completes.
This is common with slower setups (network/drives) or poor operational practices (many objects in one prefix, flat organization).
You can set a timeout value that is above what your cluster requires on average to complete an operation to address the above issue.
Place the following nginx annotations in the ingress configuration under `metadata.anotations`:


* `nginx.ingress.kubernetes.io/proxy-read-timeout`: Controls how long the NGINX load balancer waits for a response from the backend.

* `nginx.ingress.kubernetes.io/proxy-send-timeout`: Controls how long the NGINX load balancer waits to send a request to the backend.

Here is the updated ingress configuration:

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-minio
namespace: tenant1-ns
annotations:
kubernetes.io/ingress.class: "nginx"
## Remove if using CA signed certificate
nginx.ingress.kubernetes.io/proxy-ssl-verify: "off"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/proxy-body-size: "0"
nginx.ingress.kubernetes.io/server-snippet: |
client_max_body_size 0;
nginx.ingress.kubernetes.io/configuration-snippet: |
chunked_transfer_encoding off;
nginx.ingress.kubernetes.io/proxy-read-timeout: "60s"
nginx.ingress.kubernetes.io/proxy-send-timeout: "60s"
Comment on lines +162 to +163
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: Is there a value here we would consider to be too high? We have internal timeouts as well. Just curious.

spec:
tls:
- hosts:
- minio.example.com
secretName: nginx-tls
rules:
- host: minio.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: minio
port:
number: 443
```

> Explanation:

* `proxy-read-timeout`: Ensures that NGINX waits for 60 seconds for a response from the MinIO backend before timing out.

* `proxy-send-timeout`: Ensures that NGINX waits for 60 seconds for data to be sent to the MinIO backend before timing out.

Apply the updated configuration using `kubectl apply -f <file_name>.yaml`. This will set the desired timeouts for read and write operations.
Loading