Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create new IA for observability docs #4134

Merged
merged 27 commits into from
Sep 4, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
90b6484
Create new IA for observability docs
dedemorton Aug 9, 2024
de1cddf
Try making the nav titles shorter
dedemorton Aug 13, 2024
80621c7
Merge branch 'main' into issue#247
dedemorton Aug 19, 2024
89247f1
Align TOC with latest outline
dedemorton Aug 19, 2024
45527e0
Fix link text and rename analyze-metrics file
dedemorton Aug 19, 2024
6fd7dfd
Add redirect page for observability tutorials
dedemorton Aug 20, 2024
c80b2c8
Add content to start here topic
dedemorton Aug 20, 2024
fe338df
Add content to overview topic about monitoring applications and services
dedemorton Aug 20, 2024
c6aa571
Add content to overview topic about incident management
dedemorton Aug 20, 2024
2aec89f
Remove old tutorial container topic
dedemorton Aug 20, 2024
2701742
Apply changes from colleenmcginnis
dedemorton Aug 23, 2024
aafd6f1
Apply feedback from alaudazzi
dedemorton Aug 23, 2024
ed55b90
More K8s tutorial to infra/hosts monitoring section
dedemorton Aug 23, 2024
8546736
Add getting started change and other changes requested by obs docs
dedemorton Aug 24, 2024
ed321e3
Fix broken apm links
dedemorton Aug 26, 2024
a0b7a5e
Apply more link fixes
dedemorton Aug 26, 2024
5b301a8
Merge branch 'main' into issue#247
dedemorton Aug 26, 2024
b7695cb
Reorganize getting started/quick start section
dedemorton Aug 26, 2024
8b0f363
Resolve or remove related TODOs
dedemorton Aug 28, 2024
a1cd10a
Merge branch 'main' into issue#247
dedemorton Aug 28, 2024
22af269
Update docs/en/observability/splunk-get-started.asciidoc
dedemorton Aug 29, 2024
af8af2b
Make changes to getting started section
dedemorton Aug 29, 2024
8d975b0
Move universal profiling to section about infra monitoring
dedemorton Aug 30, 2024
9f54a64
Move troubleshooting up a level in the nav
dedemorton Sep 3, 2024
eedfc98
Delete old APM tutorial
dedemorton Sep 3, 2024
bbc3284
Merge branch 'main' into issue#247
dedemorton Sep 3, 2024
f8602d2
Fix jump tables
dedemorton Sep 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/en/apm-server/redirects.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -428,7 +428,7 @@ Refer to {observability-guide}/apm.html[Application Performance Monitoring (APM)

{move-notice}

Refer to {observability-guide}/traces-get-started.html[Quick start with Elastic Cloud].
Refer to {observability-guide}/get-started-with-fleet-apm-server.html[Quick start with Elastic Cloud].

[role="exclude",id="getting-started-apm-server"]
=== Self manage APM Server
Expand Down Expand Up @@ -491,7 +491,7 @@ Refer to {observability-guide}/apm-running-on-docker.html[Run APM Server on Dock

{move-notice}

Refer to {observability-guide}/_fleet_managed_apm_server.html[Fleet-managed APM Server].
Refer to {observability-guide}/get-started-with-fleet-apm-server.html[Fleet-managed APM Server].

[role="exclude",id="_step_1_set_up_fleet"]
=== Step 1: Set up Fleet
Expand All @@ -505,7 +505,7 @@ Refer to {observability-guide}/_step_1_set_up_fleet.html[Step 1: Set up Fleet].

{move-notice}

Refer to {observability-guide}/_step_2_add_and_configure_the_apm_integration.html[Step 2: Add and configure the APM integration].
Refer to {observability-guide}/add-apm-integration.html[Step 2: Add and configure the APM integration].

[role="exclude",id="_step_3_install_apm_agents"]
=== Step 3: Install APM agents
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,5 +47,5 @@ which means that other users need to turn on the new experience if they want to
To add services to the inventory, use one of the following approaches:

* Instrument your services with APM or OpenTelemetry.
To learn how, refer to <<traces-get-started>>.
To learn how, refer to <<get-started-with-fleet-apm-server>>.
* Declare `service.name` in your logs. To learn how, <<add-logs-service-name>>.
2 changes: 1 addition & 1 deletion docs/en/observability/apm.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ like JVM metrics in the Java Agent, and Go runtime metrics in the Go Agent.
[float]
=== Give Elastic APM a try

Use <<traces-get-started,Get started with application traces and APM>> to quickly spin up an APM deployment.
Use <<get-started-with-fleet-apm-server,Get started with application traces and APM>> to quickly spin up an APM deployment.
Want to host everything yourself instead? See <<apm-getting-started-apm-server>>.

include::{observability-docs-root}/docs/en/observability/apm/getting-started-apm-server.asciidoc[]
Expand Down
2 changes: 2 additions & 0 deletions docs/en/observability/apm/apm-quick-start.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
[[apm-quick-start]]
== Quick start with {ecloud}

TODO: Ask Colleen if this file should be deleted.

The easiest way to get started with Elastic APM is by using our
{ess-product}[hosted {es} Service] on {ecloud}.
The {es} Service is available on AWS, GCP, and Azure.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
TIP: The easiest way to get started with Elastic APM is by using our
{ess-product}[hosted {es} Service] on {ecloud}.
The {es} Service is available on AWS, GCP, and Azure.
See <<traces-get-started,getting started documentation>> to get started in minutes.
See <<get-started-with-fleet-apm-server,getting started documentation>> to get started in minutes.


// TODO: MOVE THIS
Expand Down Expand Up @@ -475,7 +475,7 @@ include::{observability-docs-root}/docs/en/observability/apm/repositories.asciid
// Shared docker
include::{observability-docs-root}/docs/en/observability/apm/shared-docker.asciidoc[]


[[get-started-with-fleet-apm-server]]
=== Fleet-managed APM Server

This guide will explain how to set up and configure a Fleet-managed APM Server.
Expand All @@ -498,6 +498,7 @@ include::{ingest-docs-root}/docs/en/ingest-management/tab-widgets/add-fleet-serv

For more information, refer to {fleet-guide}/fleet-server.html[{fleet-server}].

[[add-apm-integration]]
==== Step 2: Add and configure the APM integration

include::{observability-docs-root}/docs/en/observability/tab-widgets/add-apm-integration/content.asciidoc[tag=self-managed]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<titleabbrev>Advanced setup</titleabbrev>
++++

Before reading this section, see the <<traces-get-started,getting started documentation>>
Before reading this section, see the <<get-started-with-fleet-apm-server,getting started documentation>>
for basic installation and running instructions.

This section includes additional information on how to set up and run APM Server, including:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
[[application-and-service-monitoring]]
= Application and service monitoring

++++
<titleabbrev>Applications and services</titleabbrev>
++++

Explore the topics in this section to learn how to observe and monitor software applications and services running in your environment.

[cols="1,1"]
|===
|<<apm>>
|Monitor software services and applications in real time, by collecting detailed performance information on response time for incoming requests, database queries, calls to caches, external HTTP requests, and more.

|<<monitor-uptime-synthetics>>
|Monitor the availability of network endpoints and services.

|<<user-experience>>
|Quantify and analyze the perceived performance of your web application using real-world user experiences.

|<<uptime-intro>>
|Periodically check the status of your services and applications.

|<<monitor-java-app>>
|Monitor a Java application using Elastic Observability: Logs, Infrastructure metrics, APM, and Uptime.

|<<troubleshooting>>
|Troubleshoot issues across Observability apps: select and filter result data in any dimension, look for the cause or impact of performance problems, and view information about all requests that were made to collect the data displayed on the current page.
|===
6 changes: 5 additions & 1 deletion docs/en/observability/ci-cd-observability.asciidoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
[[ci-cd-observability]]
= CI/CD observability
= Continuous Integration and Continuous Delivery (CI/CD) monitoring
:figure-caption!:

++++
<titleabbrev>CI/CD</titleabbrev>
++++

To help administrators monitor and troubleshoot their CI/CD platform and help developers
increase the speed and reliability of their CI/CD pipelines, Elastic {observability}
provides visibility in Continuous Integration and Continuous Delivery (CI/CD) processes.
Expand Down
23 changes: 23 additions & 0 deletions docs/en/observability/cloud-monitoring.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
[[cloud-monitoring]]
= Cloud monitoring

++++
<titleabbrev>Cloud</titleabbrev>
++++

Explore the topics in this section to learn how to observe and monitor a broad range of resources and applications running in your cloud ecosystem.

[cols="1,1"]
|===
|<<monitor-amazon-web-services>>
|Monitor AWS services and resources, including EC2, Kinesis data streams, S3 buckets, SQS, VPC Flow Logs, CloudTrail logs, network firewall logs, WAF logs, and CloudWatch logs.

|<<monitor-azure-web-services>>
|Get visibility into your key Azure services including through secure native integration within the Azure console.

|<<monitor-gcp>>
|Track performance of your Google Cloud services, including through agentless data ingestion with Google Pub/Sub.
|===



Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
= Monitor Amazon Cloud Compute (EC2)

++++
<titleabbrev>Monitor EC2</titleabbrev>
<titleabbrev>EC2</titleabbrev>
++++

https://aws.amazon.com/ec2[Elastic Compute Cloud (Amazon EC2)] enables
Expand Down Expand Up @@ -115,7 +115,7 @@ workloads.
A burstable performance instance configured as unlimited can sustain high CPU
utilization for any period of time whenever required. T4g, T3a, and T3 instances
launch as unlimited by default. If the average CPU usage over a 24-hour period
exceeds the baseline, you incur charges for surplus credits.
exceeds the baseline, you incur charges for surplus credits.
+
The CPU surplus credit balance metric tracks the accumulated balance. The CPU
surplus credit charged measures extra credits that will result in additional
Expand Down Expand Up @@ -227,7 +227,7 @@ of related EC2 instances.
+
The Elastic EC2 integration comes with a default dashboard that shows key
metrics like status check, CPU utilization, Disk I/O, and network throughput. It
also provides filtering for region, availability zone, and machine type.
also provides filtering for region, availability zone, and machine type.

`aws.ec2.metrics.StatusCheckFailed_Instance.avg`::
This check monitors the software and network configuration of the instance.
Expand All @@ -236,7 +236,7 @@ networking or startup configuration, exhausted memory, corrupted file system,
incompatible kernel, and so on. When an instance status check fails, you
typically must address the problem yourself. You may need to reboot the instance
or make instance configuration changes. To troubleshoot instances with failed
status checks, refer to the
status checks, refer to the
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstances.html[Amazon EC2 documentation].
+
This check returns 0 (passed) if an instance passes the system status check or 1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
= Amazon Web Services (AWS) monitoring

++++
<titleabbrev>AWS monitoring</titleabbrev>
<titleabbrev>AWS</titleabbrev>
++++

Elastic Observability offers powerful monitoring solutions to keep your AWS environments reliable and efficient, providing deep insights into the performance of your applications, services, and infrastructure components.

Learn how to use the Elastic Observability solution to observe and monitor a broad range of AWS resources and applications.
Learn how to use the Elastic Observability solution to observe and monitor a broad range of AWS resources and applications.

- <<monitor-aws-elastic-agent,Monitor {aws} with {agent}>>
- <<monitor-aws,Monitor {aws} with {beats}>>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
= Monitor Amazon Kinesis data streams

++++
<titleabbrev>Monitor Kinesis data streams</titleabbrev>
<titleabbrev>Kinesis data streams</titleabbrev>
++++

https://aws.amazon.com/kinesis/data-streams/[Amazon Kinesis Data Streams]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
= Monitor Amazon Simple Storage Service (S3)

++++
<titleabbrev>Monitor S3</titleabbrev>
<titleabbrev>S3</titleabbrev>
++++

https://aws.amazon.com/s3/[Amazon Simple Storage Service (S3)]
Expand All @@ -25,7 +25,7 @@ storage. These metrics are reported once per day by default and are provided to
AWS customers at no additional cost.
* Request metrics. Use these metrics to quickly identify and act on
operational issues. These request metrics are available at one-minute intervals
after some latency for processing, and they are not enabled by default.
after some latency for processing, and they are not enabled by default.

[discrete]
[[get-started-s3]]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
= Monitor Amazon Simple Queue Service (SQS)

++++
<titleabbrev>Monitor SQS</titleabbrev>
<titleabbrev>SQS</titleabbrev>
++++

https://aws.amazon.com/sqs/[Amazon Simple Queue Service (SQS)] is a message
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -410,5 +410,4 @@ To track your AWS billing, open the
[role="screenshot"]
image::images/agent-tut-billing-dashboard.png[Screenshot of the Billing Overview dashboard]

Congratulations! You have completed the tutorial. To try other tutorials in this
series, visit the <<observability-tutorials>> page.
Congratulations! You have completed the tutorial.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
= Monitor CloudTrail logs

++++
<titleabbrev>Monitor CloudTrail logs</titleabbrev>
<titleabbrev>CloudTrail logs</titleabbrev>
++++

In this section, you'll learn how to monitor and analyze the CloudTrail logs you send to Elastic with Amazon Data Firehose. You will go through the following steps:
Expand Down Expand Up @@ -41,7 +41,7 @@ image::firehose-cloudtrail-cloudwatch.png[Cloudtrail to CloudWatch]

To export CloudTrail logs to CloudWatch, you must set up a *trail* through the following steps:

. Go to the https://console.aws.amazon.com/[AWS console] and navigate to CloudTrail.
. Go to the https://console.aws.amazon.com/[AWS console] and navigate to CloudTrail.

. Click *Create trail* and configure the general details on the *Choose trail attributes* panel, like:
+
Expand Down Expand Up @@ -91,13 +91,13 @@ For more information on how to set up a Amazon Data Firehose delivery stream to
- Elastic endpoint URL
- API key
- Content encoding: gzip
- Retry duration: 60 (default)
- Retry duration: 60 (default)
- Backup settings: failed data only to s3 bucket

You now have an Amazon Data Firehose delivery specified with:

- source: direct put
- destination: elastic
- source: direct put
- destination: elastic
- parameters: es_datastream_name: logs-aws.cloudtrail-default

[discrete]
Expand All @@ -106,7 +106,7 @@ You now have an Amazon Data Firehose delivery specified with:

image::firehose-subscription-filter.png[Firehose subscription filter]

The Amazon Data Firehose delivery stream is ready to send logs to your Elastic Cloud deployment.
The Amazon Data Firehose delivery stream is ready to send logs to your Elastic Cloud deployment.

. Visit the log group with the CloudTrail events.
+
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@
= Monitor CloudWatch logs

++++
<titleabbrev>Monitor CloudWatch logs</titleabbrev>
<titleabbrev>CloudWatch logs</titleabbrev>
++++

In this section, you'll learn how to export log events from CloudWatch logs to an Elastic cluster by using Amazon Data Firehose.

You'll go through the following steps:

- Install AWS integration in {kib}
- Select a CloudWatch log group to monitor
- Select a CloudWatch log group to monitor
- Create a delivery stream in Amazon Data Firehose
- Set up a subscription filter to forward the logs using the Firehose stream
- Visualize your logs in {kib}
Expand All @@ -36,17 +36,17 @@ IMPORTANT: AWS PrivateLink is not supported. Make sure the deployment is on AWS,

[discrete]
[[firehose-cloudwatch-step-two]]
== Step 2: Select a CloudWatch log group to monitor
== Step 2: Select a CloudWatch log group to monitor

image::firehose-cloudwatch-log-group.png[CloudWatch log group]

In this tutorial, you are going to collect application logs from an AWS Lambda-based app and forward them to Elastic.
In this tutorial, you are going to collect application logs from an AWS Lambda-based app and forward them to Elastic.

**Create a Lambda function**

NOTE: You can skip this section if you already have a Lambda function, or any other service or application that sends logs to a CloudWatch log group. Take note of the log group from which you want to collect log events and move to the next section.
NOTE: You can skip this section if you already have a Lambda function, or any other service or application that sends logs to a CloudWatch log group. Take note of the log group from which you want to collect log events and move to the next section.

Like many other services and platforms in AWS, Lambda functions natively log directly to CloudWatch out of the box.
Like many other services and platforms in AWS, Lambda functions natively log directly to CloudWatch out of the box.

. Go to the https://console.aws.amazon.com/[AWS console] and open the AWS Lambda page.
. Click **Create function** and select the option to create a function from scratch.
Expand Down Expand Up @@ -90,11 +90,11 @@ Take note of the log group name for this Lambda function, as you will need it in

image::firehose-cloudwatch-firehose-stream.png[Amazon Firehose Stream]

. Go to the https://console.aws.amazon.com/[AWS console] and navigate to Amazon Data Firehose.
. Go to the https://console.aws.amazon.com/[AWS console] and navigate to Amazon Data Firehose.

. Click *Create Firehose stream* and choose the source and destination of your Firehose stream. Unless you are streaming data from Kinesis Data Streams, set source to `Direct PUT` and destination to `Elastic`.
. Click *Create Firehose stream* and choose the source and destination of your Firehose stream. Unless you are streaming data from Kinesis Data Streams, set source to `Direct PUT` and destination to `Elastic`.

. Provide a meaningful *Firehose stream name* that will allow you to identify this delivery stream later.
. Provide a meaningful *Firehose stream name* that will allow you to identify this delivery stream later.
+
NOTE: For advanced use cases, source records can be transformed by invoking a custom Lambda function. When using Elastic integrations, this should not be required.

Expand All @@ -111,7 +111,7 @@ image::firehose-cloudwatch-subscription-filter.png[CloudWatch subscription filte

To send log events from CloudWatch to Firehose, open the log group where the Lambda service is logging and create a subscription filter.

**Create a subscription filter for Amazon Data Firehose**
**Create a subscription filter for Amazon Data Firehose**

The https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Subscriptions.html[subscription filter] allows you to pick log events from the log group and forward them to other services, such as an Amazon Kinesis stream, an Amazon Data Firehose stream, or AWS Lambda.

Expand Down Expand Up @@ -188,8 +188,8 @@ image::firehose-cloudwatch-destination-errors.png[Firehose destination errors]

The Firehose delivery stream reports:

* The number of failed deliveries.
* The failure detail.
* The number of failed deliveries.
* The failure detail.

[discrete]
[[firehose-cloudwatch-step-five]]
Expand Down
Loading