diff --git a/docs/en/observability/monitor-nginx-ml.asciidoc b/docs/en/observability/monitor-nginx-ml.asciidoc new file mode 100644 index 0000000000..e07e24808d --- /dev/null +++ b/docs/en/observability/monitor-nginx-ml.asciidoc @@ -0,0 +1,70 @@ +[discrete] +[[monitor-nginx-ml]] +== Part 5: Find anomalies in your nginx access logs + +Use the {integrations-docs}/nginx[nginx Elastic integration] machine learning (ML) module to help find unusual activity in your nginx access logs. +Monitoring anomalies in your access logs helps you detect: + +* security threats +* network issues +* system performance issues +* operational efficiency issues + +[discrete] +[[monitor-nginx-ml-jobs]] +=== nginx anomaly detection jobs + +The nginx ML module provides the following anomaly detection jobs: + +[[horizontal]] +Low request rates (`low_request_rate_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-count[`low_count`] function to detect abnormally low request rates. Abnormally low request rates might indicate that network issues or other issues are preventing requests from reaching the server. +Unusual source IPs - high request rates (`source_ip_request_rate_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-count[`hight_count`] function to detect abnormally high request rates from individual IP addresses. Many requests from a single IP or small group of IPs might indicate something malicious like a distributed denial of service (DDoS) attack where a large number of requests are sent to overwhelm the server and make it unavailable to users. +Unusual source IPs - high distinct count of URLs (`source_ip_url_count_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-distinct-count[`high_distinct_count`] function to detect individual IP addresses accessing abnormally high numbers of unique URLs. A single IP accessing many unique URLs might indicate something malicious like web scraping or an attempt to find sensitive data or vulnerabilities. +Unusual status code rates (`status_code_rate_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-count[`count`] function to detect abnormal error status code rates. A high rate of status codes could indicate problems with broken links, bad URLs, or unauthorized access attempts. A high rate of status codes could also point to server issues like limited resources or bugs in your code. +Unusual visitor rates (`visitor_rate_nginx`):: Uses the {ml-docs}/ml-count-functions.html#ml-nonzero-count[`non_zero_count`] function to detect abnormal visitor rates. High visitor rates could indicate something malicious like a DDoS attack. +Low visitor rates could indicate issues with access to the server. + +NOTE: These anomaly detection jobs are available when you have data that matches the query specified in the ML module manifest. Users not following this tutorial can refer to {integrations-docs}/nginx#ml-modules[nginx integration ML modules] for more about the ML module manifest. + +[discrete] +[[monitor-nginx-ml-prereqs]] +=== Before you begin + +Verify that your environment is set up properly to use the {ml-features}. +If {es} {security-features} are enabled, you need a user with permissions to manage {anomaly-jobs}. +Refer to {ml-docs}/setup.html[Set up ML features]. + +[discrete] +[[monitor-nginx-ml-add-jobs]] +=== Add nginx ML jobs + +Add the nginx ML jobs from the nginx integration to start using anomaly detection: + +. From the main {kib} menu, go to *Machine Learning*. Under *Anomaly Detection*, select *Jobs*. +. Select *Create job*. +. In the search bar, enter *nginx* and select *Nginx access logs [Logs Nginx]*. +. Under *Use preconfigured jobs*, select the *Nginx access logs* card. +. Select *Create jobs*. + +Back on the *Anomaly Detection Jobs* page, you should see the nginx anomaly detection jobs—`low_request_rate_nginx`, `source_ip_request_rate_nginx`, `source_ip_url_count_nginx`, `status_code_rate_nginx`, and `visitor_rate_nginx`. + +[discrete] +[[monitor-nginx-ml-explore]] +=== Explore your anomaly detection job results + +View your anomaly detection job results using the Anomaly Explorer or Single Metric Viewer found under *Anomaly Detection* in the Machine Learning menu. +The Anomaly Explorer shows the results from all or any combination of your nginx ML jobs. +The Single Metric Viewer focuses on a specific job. +These tools offer a comprehensive view of anomalies and help find patterns and irregularities across data points and time intervals. + +Refer to {ml-docs}/ml-ad-view-results.html[View anomaly detection job results] for more on viewing and understanding your anomaly detection job results. + +[discrete] +[[monitor-nginx-ml-alert]] +=== Set up alerts + +With the nginx ML jobs detecting anomalies, you can set rules to generate alerts when your jobs meet specific conditions. +For example, you could set up a rule on the `low_request_rate_nginx` job to alert when low request rates hit a specific severity threshold. +When you get alerted, you can make sure your server isn't experiencing issues. + +Refer to {ml-docs}/ml-configuring-alerts.html[Generating alerts for anomaly detection jobs] for more on setting these rules and generating alerts. \ No newline at end of file diff --git a/docs/en/observability/monitor-nginx.asciidoc b/docs/en/observability/monitor-nginx.asciidoc index b9225d3e78..e4d7a4dedc 100644 --- a/docs/en/observability/monitor-nginx.asciidoc +++ b/docs/en/observability/monitor-nginx.asciidoc @@ -9,7 +9,7 @@ Return to this tutorial after you've learned the basics. **** -Use the https://docs.elastic.co/integrations/nginx[Nginx Elastic integration] and the {agent} to collect valuable metrics and logs from your nginx instances. Then, use built-in dashboards and tools like Logs Explorer in {kib} allow you to visualize and monitor your nginx data from one place. This data provides valuable insight into your nginx instances—for example: +Use the {integrations-docs}/nginx[nginx Elastic integration] and the {agent} to collect valuable metrics and logs from your nginx instances. Then, use built-in dashboards and tools like Logs Explorer in {kib} to visualize and monitor your nginx data from one place. This data provides valuable insight into your nginx instances—for example: * A spike in error logs for a certain resource may mean you have a deleted resource that is still needed. * Access logs can show when a service's peak times are, and, from this, when it might be best to perform things like maintenance. @@ -199,7 +199,7 @@ To open the nginx dashboard: The *Metrics Nginx overview* shows visual representations of total requests, processed requests, heartbeat/up, active connections, reading/writing/waiting rates, request rate, accepts and handled rates, and drops rate. [role="screenshot"] -image::images/nginx-metrics-dashboard.png[Nginx metrics dashboard, 75%] +image::images/nginx-metrics-dashboard.png[nginx metrics dashboard, 75%] [discrete] [[monitor-nginx-explore-logs]] @@ -229,7 +229,7 @@ image::images/nginx-logs-explorer.png[Logs Explorer showing nginx error logs] [discrete] [[monitor-nginx-logs-dashboard]] -==== Nginx logs dashboards +==== nginx logs dashboards The nginx integration has built-in dashboards that show the full picture of your nginx logs in one place. To open the nginx dashboards: @@ -246,4 +246,6 @@ image::images/nginx-logs-overview-dashboard.png[nginx logs overview dashboard, 7 The *Nginx access and error logs* dashboard shows your access logs over time, and lists your access and error logs. [role="screenshot"] -image::images/nginx-logs-access-error-dashboard.png[nginx access and error logs dashboard, 75%] \ No newline at end of file +image::images/nginx-logs-access-error-dashboard.png[nginx access and error logs dashboard, 75%] + +include::monitor-nginx-ml.asciidoc[] \ No newline at end of file