Skip to content

Commit

Permalink
Merge pull request #2 from mdbirnstiehl/otel-acronym-update
Browse files Browse the repository at this point in the history
Use EDOT acronym
  • Loading branch information
mlunadia authored Aug 13, 2024
2 parents f250533 + 6761d08 commit aa35cf1
Show file tree
Hide file tree
Showing 9 changed files with 49 additions and 42 deletions.
26 changes: 14 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,29 @@
# Elastic Distributions for OpenTelemetry
# Elastic Distributions of OpenTelemetry

[OpenTelemetry](https://opentelemetry.io/docs/) is a vendor-neutral observability framework for collecting, processing, and exporting telemetry data.
The Elastic Distribution for OpenTelemetry Collector (Elastic OTel Collector) is a supported, drop-in replacement distribution of the [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector) made up of mostly upstream components.
You can send your telemetry data to Elastic Observability using OpenTelemetry the following ways:

- Collect and send logs and host metrics to [Elastic Cloud](https://cloud.elastic.co/) using the Elastic OTel Collector.
- Instrument your applications and send logs, traces, and metrics to [Elastic Cloud](https://cloud.elastic.co/) using the Elastic Distributions for select [OpenTelemetry language SDKs](https://opentelemetry.io/docs/languages/). Currently, Elastic provides distributions for the following language SDKs: Java, .NET, Node.js, and Python.
The Elastic Distributions of OpenTelemetry (EDOT) are composed of OpenTelemetry (OTel) project components, [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector), and language SDKs, which provide users with the necessary capabilities and out-of-the-box configurations, enabling quick and effortless infrastructure and application monitoring.

You can use OpenTelemetry to send your telemetry data to Elastic Observability in the following ways:

- Collect and send logs and host metrics to [Elastic Cloud](https://cloud.elastic.co/) using the EDOT Collector.
- Instrument your applications and send logs, traces, and metrics to [Elastic Cloud](https://cloud.elastic.co/) using the Elastic Distributions of select [OpenTelemetry language SDKs](https://opentelemetry.io/docs/languages/). Currently, Elastic provides distributions for the following language SDKs: Java, .NET, Node.js, and Python.
- Build and configure a [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or extend the [OpenTelemetry Collector Contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution to collect logs and metrics and send them to Elastic Observability.

This diagram provides a quick overview on how the different components work together. Refer to the [components](docs/collector-components.md) for a more in-depth look.
This diagram provides a quick overview of how the different components work together. Refer to the [components](docs/collector-components.md) for a more in-depth look.

![Diagram of the OpenTelemetry flow](docs/images/elastic-otel-overview.png)

## Collect infrastructure data with the Elastic Distribution for OpenTelemetry Collector
## Collect infrastructure data using the EDOT Collector

These pages detail the components and how to configure the Elastic OTel Collector.
These pages detail the components and how to configure the EDOT Collector.

- [Components](docs/collector-components.md): Get details on the components used to receive, process, and export telemetry data.
- [Guided onboarding](docs/guided-onboarding.md): Use the guided onboarding in Kibana or in a serverless Observability project to send data using the Elastic OTel Collector.
- [Manual configurations](docs/manual-configuration.md): Manually configure the Elastic OTel Collector to send data to Elastic Observability.
- [Limitations](docs/collector-limitations.md): Understand the current limitations of the Elastic OTel Collector.
- [Guided onboarding](docs/guided-onboarding.md): Use the guided onboarding in Elasticsearch Service or a serverless Observability project to send data using the EDOT Collector.
- [Manual configurations](docs/manual-configuration.md): Manually configure the EDOT Collector to send data to Elastic Observability.
- [Limitations](docs/collector-limitations.md): Understand the current limitations of the EDOT Collector.

## Collect application data with Elastic Distributions for OpenTelemetry language SDKs
## Collect application data using the EDOT language SDKs

Elastic offers several Distributions that extend [OpenTelemetry language SDKs](https://opentelemetry.io/docs/languages/). The following languages are currently supported:

Expand Down
8 changes: 4 additions & 4 deletions docs/collector-components.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Elastic Distribution for OpenTelemetry Collector components
# Elastic Distribution of OpenTelemetry Collector components

The OpenTelemetry Collector uses the following components to receive, process, and export telemetry data:

Expand All @@ -7,7 +7,7 @@ The OpenTelemetry Collector uses the following components to receive, process, a
- [Exporters](collector-components.md#exporters): Send data to the backends or destinations.
- [Extensions](collector-components.md#extensions): Provide additional functionalities and capabilities.

The default configurations of the Elastic Distribution for the OpenTelemetry Collector follows these flows.
The default configurations of the Elastic Distribution of the OpenTelemetry (EDOT) Collector follows these flows:

**MacOS and Linux Host metrics:**

Expand Down Expand Up @@ -84,9 +84,9 @@ flowchart LR
style six fill: #e6f9f7,stroke:#333,stroke-width:1px
```

Refer to the following tables for more information on the components included in the Elastic OTel Collector.
Refer to the following tables for more information on the components included in the EDOT Collector.
Follow the links for OpenTelemetry documentation with more configuration details for each component.
To set up the Elastic OTel Collector, get started using the [guided onboarding](guided-onboarding.md) docs or the [manual configuration](manual-configuration.md) docs.
To set up the EDOT Collector, get started using the [guided onboarding](guided-onboarding.md) docs or the [manual configuration](manual-configuration.md) docs.

## Receivers

Expand Down
4 changes: 2 additions & 2 deletions docs/collector-limitations.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Elastic Distribution for OpenTelemetry Collector limitations
# Elastic Distribution of OpenTelemetry Collector limitations

The Elastic Distribution for the OpenTelemetry Collector has the following limitations:
The Elastic Distribution of the OpenTelemetry Collector has the following limitations:

- Because of an upstream limitation, `host.network.*` metrics aren't present from the OpenTelemetry side.
- `process.state` isn't present in the OpenTelemetry host metric. It's set to a dummy value of **Unknown** in the **State** column of the host processes table.
Expand Down
4 changes: 2 additions & 2 deletions docs/configure-upstream-collector.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

You can build and configure a [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or extend the [OpenTelemetry Collector Contrib ](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution to collect logs and metrics and send them to Elastic Observability.

For a more seamless experience, use the Elastic Distribution for the OpenTelemetry collector.
Refer to the [guided onboarding](guided-onboarding.md) docs or the [manual configuration](manual-configuration.md) docs for more on configuring the Elastic OTel collector.
For a more seamless experience, use the Elastic Distribution of the OpenTelemetry Collector.
Refer to the [guided onboarding](guided-onboarding.md) docs or the [manual configuration](manual-configuration.md) docs for more on configuring the Elastic Distribution of the OpenTelemetry Collector.

## Upstream collector configuration examples
Refer to the OpenTelemetry documentation on [building a custom collector](https://opentelemetry.io/docs/collector/custom-collector/) for more on creating an upstream collector.
Expand Down
22 changes: 12 additions & 10 deletions docs/guided-onboarding.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,17 @@
# Collect logs and metrics using the guided onboarding
The guided onboarding in Kibana or in a serverless Observability project walks you through collecting logs and metrics using the Elastic Distribution for OpenTelemetry Collector.
To configure the Elastic OTel Collector manually, refer to the [manual configuration](manual-configuration.md) docs.
The guided onboarding in Elasticsearch Service or a serverless Observability project walks you through collecting logs and metrics using the Elastic Distribution of OpenTelemetry (EDOT) Collector.
To configure the EDOT Collector manually, refer to the [manual configuration](manual-configuration.md) docs.

## Before you begin
The onboarding has the following requirements and limitations:
The onboarding has the following requirements:

- The **Admin** role or higher is required to onboard system logs and metrics. To learn more, refer to [Assign user roles and privileges](https://www.elastic.co/docs/current/serverless/general/assign-user-roles).
- Root privileges on the host are required to run the OpenTelemetry collector used in this quickstart.
- The Elastic OTel Collector only works on Kubernetes, Linux, and MacOS systems.
- Refer to [Elastic OpenTelemetry Collector limitations](collector-limitations.md) for known limitations when using the Elastic OTel Collector.
- The guided onboarding provides out-of-the-box deployment and configurations for Kubernetes, Linux, and MacOS systems.

## Collect your logs and metrics

Follow these steps to collect logs and metrics using the Elastic OTel collector:
Follow these steps to collect logs and metrics using the EDOT Collector:

1. Open an [Elastic Cloud](cloud.elastic.co) deployment or a serverless Observability project.
1. To open the guided onboarding, either:
Expand All @@ -23,10 +22,13 @@ Follow these steps to collect logs and metrics using the Elastic OTel collector:
1. For **MacOS and Linux**, copy the command, open a terminal on your host, and run the command to download and configure the OpenTelemetry collector.
1. For **Kubernetes**, download the manifest.
1. Copy the command under Step 2:
1. For **MacOS and Linux**, run the command in your terminal to start the Elastic OTel collector.
1. For **Kubernetes**, run the command from the directory where you downloaded the manifest to install the Elastic OTel Collector on every node of your cluster.
1. For **MacOS and Linux**, run the command in your terminal to start the EDOT Collector.
1. For **Kubernetes**, run the command from the directory where you downloaded the manifest to install the EDOT Collector on every node of your cluster.

Logs are collected from setup onward, so you won't see logs that occurred before starting the Elastic OTel Collector.
Logs are collected from setup onward, so you won't see logs that occurred before starting the EDOT Collector.
The default log path is `/var/log/*`. To update the path, modify `otel.yml`.

Under **Visualize your data**, you'll see links to **Logs Explorer** to view your logs and **Hosts** to view your host metrics.
Under **Visualize your data**, you'll see links to **Logs Explorer** to view your logs and **Hosts** to view your host metrics.

## Limitations
Refer to [Elastic OpenTelemetry Collector limitations](collector-limitations.md) for known limitations when using the EDOT Collector.
Binary file modified docs/images/elastic-otel-overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/images/infrastructure-elastic-otel-flow.png
Binary file not shown.
Binary file removed docs/images/kubernetes-elastic-otel-flow.png
Binary file not shown.
27 changes: 15 additions & 12 deletions docs/manual-configuration.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,40 @@
# Manually configure the Elastic Distribution for OpenTelemetry Collector
Collecting logs and host metrics with the Elastic Distribution for OpenTelemetry Collector without using the [guided onboarding](guided-onboarding.md) requires some manual configuration.
# Manually configure the Elastic Distribution of OpenTelemetry Collector
Collecting logs and host metrics with the Elastic Distribution of OpenTelemetry (EDOT) Collector without using the [guided onboarding](guided-onboarding.md) requires completing some manual steps.

## Before you begin
The Elastic OTel collector has the following requirements and limitations:
The EDOT Collector has the following requirements:

- The **Admin** role or higher is required to onboard system logs and metrics. To learn more, refer to [Assign user roles and privileges](https://www.elastic.co/docs/current/serverless/general/assign-user-roles).
- Root privileges on the host are required to run the OpenTelemetry collector used in this quickstart.
- The Elastic OTel Collector only works on Kubernetes, Linux, and MacOS systems.
- Refer to [Elastic OpenTelemetry Collector limitations](collector-limitations.md) for known limitations when using the Elastic Distribution for the OpenTelemetry collector.
- The EDOT Collector only works on Kubernetes, Linux, and MacOS systems.

## Collect your logs and metrics

To manually configure the Elastic OTel Collector, gather the following information:
To manually configure the EDOT Collector, gather the following information:

- **Your Elasticsearch endpoint**: From the help menu in Kibana or your serverless Observability project, select **Connection details** and copy the **Elasticsearch endpoint**.
- **Your Elasticsearch endpoint**: From the help menu in Elastic Cloud or your serverless Observability project, select **Connection details** and copy the **Elasticsearch endpoint**.
- **API key**:
- **Kibana:** From the help menu, select **Connection details** and select **Create and manage API keys**. From the **API keys** page, select **Create API key**. Give your API key a name, select **Create API key**, and copy the new API key.
- **Elasticsearch Service:** From the help menu, select **Connection details** and select **Create and manage API keys**. From the **API keys** page, select **Create API key**. Give your API key a name, select **Create API key**, and copy the new API key.
- **Serverless:** From the help menu, select **Connection details** and select the **API key** tab. Give your API key a name, select **Create API key**, and copy the new API key.

Then manually configure the Elastic OTel Collector to collect logs and metrics on a MacOS or Linux system:
Then manually configure the EDOT Collector to collect logs and metrics on a MacOS or Linux system:

1. Download and extract the standalone Elastic Agent for your platform. For more on downloading and extracting a standalone Elastic Agent, refer to the first step in [Install standalone Elastic Agents](https://www.elastic.co/guide/en/fleet/current/install-standalone-elastic-agent.html).
1. From the Elastic Agent base directory, go to the `otel_samples` directory. The `platformlogs_hostmetrics.yml` file has the configurations for the receivers, processors, and exporters needed to collect logs and host metrics.
1. Copy the content of the `platformlogs_hostmetrics.yml` file.
1. From the Elastic Agent base directory, open the `otel.yml` file, and replace the content with the copied content from `platformlogs_hostmetrics.yml`.
1. Find and update the following settings in the configuration:
- `file_storage.directory`: Set to the directory where you want to store you OpenTelemetry data. <!-- do we want to recommend a specific folder for this? -->
- `file_storage.directory`: Set to the directory where you want to store you OpenTelemetry data.
- `elasticsearch.endpoint`: Set to your Elasticsearch endpoint you copied earlier.
- `elasticsearch.api_key`: Set to the API key you created earlier.
1. Run the OpenTelemetry collector with the following command:
1. Run the EDOT collector with the following command:
```console
./elastic-agent otel --config otel.yml
```

Logs are collected from setup onward, so you won't see logs that occurred before starting the collector.
The default log path is `/var/log/*`. Update the path in the `otel.yml` file.
The default log path is `/var/log/*`. Update the path in the `otel.yml` file.

## Limitations

Refer to [Elastic OpenTelemetry Collector limitations](collector-limitations.md) for known limitations when using the Elastic Distribution of the OpenTelemetry collector.

0 comments on commit aa35cf1

Please sign in to comment.