From 39ffc50ce1cfd89bcc64f5a3210b7d1af426c2f8 Mon Sep 17 00:00:00 2001
From: "mergify[bot]" <37929162+mergify[bot]@users.noreply.github.com>
Date: Thu, 30 May 2024 21:16:09 +0000
Subject: [PATCH] add nginx ML module tutorial (#3878) (#3951)

(cherry picked from commit bc6d9b51a52dc1ea8dad48499c10ef086e8efb36)

Co-authored-by: Mike Birnstiehl <114418652+mdbirnstiehl@users.noreply.github.com>
---
 .../observability/monitor-nginx-ml.asciidoc   | 70 +++++++++++++++++++
 docs/en/observability/monitor-nginx.asciidoc  | 10 +--
 2 files changed, 76 insertions(+), 4 deletions(-)
 create mode 100644 docs/en/observability/monitor-nginx-ml.asciidoc

diff --git a/docs/en/observability/monitor-nginx-ml.asciidoc b/docs/en/observability/monitor-nginx-ml.asciidoc
new file mode 100644
index 0000000000..e07e24808d
--- /dev/null
+++ b/docs/en/observability/monitor-nginx-ml.asciidoc
@@ -0,0 +1,70 @@
+[discrete]
+[[monitor-nginx-ml]]
+== Part 5: Find anomalies in your nginx access logs
+
+Use the {integrations-docs}/nginx[nginx Elastic integration] machine learning (ML) module to help find unusual activity in your nginx access logs.
+Monitoring anomalies in your access logs helps you detect:
+
+* security threats
+* network issues
+* system performance issues
+* operational efficiency issues
+
+[discrete]
+[[monitor-nginx-ml-jobs]]
+=== nginx anomaly detection jobs
+
+The nginx ML module provides the following anomaly detection jobs:
+
+[[horizontal]]
+Low request rates (`low_request_rate_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-count[`low_count`] function to detect abnormally low request rates. Abnormally low request rates might indicate that network issues or other issues are preventing requests from reaching the server.
+Unusual source IPs - high request rates (`source_ip_request_rate_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-count[`hight_count`] function to detect abnormally high request rates from individual IP addresses. Many requests from a single IP or small group of IPs might indicate something malicious like a distributed denial of service (DDoS) attack where a large number of requests are sent to overwhelm the server and make it unavailable to users.
+Unusual source IPs - high distinct count of URLs (`source_ip_url_count_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-distinct-count[`high_distinct_count`] function to detect individual IP addresses accessing abnormally high numbers of unique URLs. A single IP accessing many unique URLs might indicate something malicious like web scraping or an attempt to find sensitive data or vulnerabilities.
+Unusual status code rates (`status_code_rate_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-count[`count`] function to detect abnormal error status code rates. A high rate of status codes could indicate problems with broken links, bad URLs, or unauthorized access attempts. A high rate of status codes could also point to server issues like limited resources or bugs in your code.
+Unusual visitor rates (`visitor_rate_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-nonzero-count[`non_zero_count`] function to detect abnormal visitor rates. High visitor rates could indicate something malicious like a DDoS attack.
+Low visitor rates could indicate issues with access to the server.
+
+NOTE: These anomaly detection jobs are available when you have data that matches the query specified in the ML module manifest. Users not following this tutorial can refer to {integrations-docs}/nginx#ml-modules[nginx integration ML modules] for more about the ML module manifest.
+
+[discrete]
+[[monitor-nginx-ml-prereqs]]
+=== Before you begin
+
+Verify that your environment is set up properly to use the {ml-features}.
+If {es} {security-features} are enabled, you need a user with permissions to manage {anomaly-jobs}.
+Refer to {ml-docs}/setup.html[Set up ML features].
+
+[discrete]
+[[monitor-nginx-ml-add-jobs]]
+=== Add nginx ML jobs
+
+Add the nginx ML jobs from the nginx integration to start using anomaly detection:
+
+. From the main {kib} menu, go to *Machine Learning*. Under *Anomaly Detection*, select *Jobs*.
+. Select *Create job*.
+. In the search bar, enter *nginx* and select *Nginx access logs [Logs Nginx]*.
+. Under *Use preconfigured jobs*, select the *Nginx access logs* card.
+. Select *Create jobs*.
+
+Back on the *Anomaly Detection Jobs* page, you should see the nginx anomaly detection jobs—`low_request_rate_nginx`, `source_ip_request_rate_nginx`, `source_ip_url_count_nginx`, `status_code_rate_nginx`, and `visitor_rate_nginx`.
+
+[discrete]
+[[monitor-nginx-ml-explore]]
+=== Explore your anomaly detection job results
+
+View your anomaly detection job results using the Anomaly Explorer or Single Metric Viewer found under *Anomaly Detection* in the Machine Learning menu.
+The Anomaly Explorer shows the results from all or any combination of your nginx ML jobs.
+The Single Metric Viewer focuses on a specific job.
+These tools offer a comprehensive view of anomalies and help find patterns and irregularities across data points and time intervals.
+
+Refer to {ml-docs}/ml-ad-view-results.html[View anomaly detection job results] for more on viewing and understanding your anomaly detection job results.
+
+[discrete]
+[[monitor-nginx-ml-alert]]
+=== Set up alerts
+
+With the nginx ML jobs detecting anomalies, you can set rules to generate alerts when your jobs meet specific conditions.
+For example, you could set up a rule on the `low_request_rate_nginx` job to alert when low request rates hit a specific severity threshold.
+When you get alerted, you can make sure your server isn't experiencing issues.
+
+Refer to {ml-docs}/ml-configuring-alerts.html[Generating alerts for anomaly detection jobs] for more on setting these rules and generating alerts.
\ No newline at end of file
diff --git a/docs/en/observability/monitor-nginx.asciidoc b/docs/en/observability/monitor-nginx.asciidoc
index b9225d3e78..e4d7a4dedc 100644
--- a/docs/en/observability/monitor-nginx.asciidoc
+++ b/docs/en/observability/monitor-nginx.asciidoc
@@ -9,7 +9,7 @@
 Return to this tutorial after you've learned the basics.
 ****
 
-Use the https://docs.elastic.co/integrations/nginx[Nginx Elastic integration] and the {agent} to collect valuable metrics and logs from your nginx instances. Then, use built-in dashboards and tools like Logs Explorer in {kib} allow you to visualize and monitor your nginx data from one place. This data provides valuable insight into your nginx instances—for example:
+Use the {integrations-docs}/nginx[nginx Elastic integration] and the {agent} to collect valuable metrics and logs from your nginx instances. Then, use built-in dashboards and tools like Logs Explorer in {kib} to visualize and monitor your nginx data from one place. This data provides valuable insight into your nginx instances—for example:
 
 * A spike in error logs for a certain resource may mean you have a deleted resource that is still needed.
 * Access logs can show when a service's peak times are, and, from this, when it might be best to perform things like maintenance.
@@ -199,7 +199,7 @@ To open the nginx dashboard:
 The *Metrics Nginx overview* shows visual representations of total requests, processed requests, heartbeat/up, active connections, reading/writing/waiting rates, request rate, accepts and handled rates, and drops rate.
 
 [role="screenshot"]
-image::images/nginx-metrics-dashboard.png[Nginx metrics dashboard, 75%]
+image::images/nginx-metrics-dashboard.png[nginx metrics dashboard, 75%]
 
 [discrete]
 [[monitor-nginx-explore-logs]]
@@ -229,7 +229,7 @@ image::images/nginx-logs-explorer.png[Logs Explorer showing nginx error logs]
 
 [discrete]
 [[monitor-nginx-logs-dashboard]]
-==== Nginx logs dashboards
+==== nginx logs dashboards
 
 The nginx integration has built-in dashboards that show the full picture of your nginx logs in one place.
 To open the nginx dashboards:
@@ -246,4 +246,6 @@ image::images/nginx-logs-overview-dashboard.png[nginx logs overview dashboard, 7
 The *Nginx access and error logs* dashboard shows your access logs over time, and lists your access and error logs.
 
 [role="screenshot"]
-image::images/nginx-logs-access-error-dashboard.png[nginx access and error logs dashboard, 75%]
\ No newline at end of file
+image::images/nginx-logs-access-error-dashboard.png[nginx access and error logs dashboard, 75%]
+
+include::monitor-nginx-ml.asciidoc[]
\ No newline at end of file