docs: add "Architecture and Load Balancers" #2574
Conversation
docs/architecture/lb/nginx-os.md
Outdated
|
|
||
| # Define a common configuration block for all servers | ||
| map $server_port $upstream_name { | ||
| 514 stream_syslog_514; |
There was a problem hiding this comment.
Probably we also need tell that we can use port segregation, it will increase performance:
SC4S_LISTEN_DEFAULT_TCP_PORT=514,515,516,517
|
@mstopa-splunk @rjha-splunk Are we have a plan to create ansible playbook that will bootstrap sc4s instances with LB? |
@ikheifets-splunk not yet |
|
|
||
| The most reliable way to gather syslog traffic is through edge collection rather than centralized collection. When the syslog server is centrally located, UDP and stateless TCP traffic cannot adapt, leading to potential data loss. | ||
|
|
||
| For optimal reliability, deploy SC4S instances in the same VLAN as the source devices. |
There was a problem hiding this comment.
Deploy SC4S instances in the same VLAN as the source devices.
|
|
||
| ## Avoid Load Balancing for Syslog | ||
|
|
||
| For optimal performance, scale vertically by fine-tuning a single, robust server. Key tools and methods for enhancing performance on your SC4S server are documented in: |
There was a problem hiding this comment.
Scale vertically by fine-tuning a single, robust server. Tools and methods for enhancing performance on your SC4S server are documented in:
| 1. [Fine-tune for TCP](tcp-optimization.md) | ||
| 2. [Fine-tune for UDP](udp-optimization.md) | ||
|
|
||
| We advise against co-locating syslog-ng servers for horizontal scaling with load balancers. The challenges of load balancing for horizontal scaling are outlined in the [Load Balancer's Overview](lb/index.md) section. |
There was a problem hiding this comment.
Avoid co-locating syslog-ng servers for horizontal scaling with load balancers. Load balancing challenges for horizontal scaling are described in the Load Balancer's Overview section.
|
|
||
| ## High Availability (HA) Considerations | ||
|
|
||
| Syslog, being prone to data loss, can only achieve "mostly available" data collection. |
There was a problem hiding this comment.
Syslog is prone to data loss and can only achieve "mostly available" data collection.
|
|
||
| ### HA Without Load Balancers | ||
|
|
||
| Load balancing does not suit syslog’s stateless, unacknowledged traffic. More data is preserved with simpler designs, such as vMotioned VMs. |
There was a problem hiding this comment.
Load balancing does not work well with syslog’s stateless, unacknowledged traffic. To preserve more data by using simpler designs, such as vMotioned VMs.
|
|
||
| Consider applying these changes to your infrastructure. After each adjustment, run the [performance tests](performance-tests.md#check-your-udp-performance) and retain the changes that result in improvements. | ||
|
|
||
| ## Tune Your Receive Buffer |
There was a problem hiding this comment.
Tune your receiving buffer
| sudo sysctl -p | ||
| ``` | ||
|
|
||
| 2. Update `/opt/sc4s/env_file` |
There was a problem hiding this comment.
- Update
/opt/sc4s/env_file:
| SC4S_SOURCE_UDP_SO_RCVBUFF=536870912 | ||
| ``` | ||
|
|
||
| 3. Restart SC4S |
|
|
||
| 3. Restart SC4S | ||
|
|
||
| ## Tune UDP Fetch Limit |
There was a problem hiding this comment.
Tune UDP fetch limit
| SC4S_SOURCE_UDP_FETCH_LIMIT=1000000 | ||
| ``` | ||
|
|
||
| ## Increase the Number of UDP Sockets |
There was a problem hiding this comment.
Increase the number of UDP sockets
No description provided.