Skip to content

Commit

Permalink
Directory re-org
Browse files Browse the repository at this point in the history
  • Loading branch information
rcastley committed Jan 30, 2025
1 parent 6e6fb5d commit 21d3ced
Show file tree
Hide file tree
Showing 19 changed files with 80 additions and 62 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,11 @@ Next, check the agent’s debug console in the `Agent` terminal window. You shou

```text
2025-01-13T13:26:13.502+0100 info Traces {"kind": "exporter", "data_type": "traces", "name": "debug", "resource spans": 1, "spans": 1}
```

{{% expand "View Full Debug Log" %}}

```text
2025-01-13T13:26:13.502+0100 info ResourceSpans #0
Resource SchemaURL:
Resource attributes:
Expand Down Expand Up @@ -222,5 +227,6 @@ Attributes:
-> user.mastercard: Str(5555 5555 5555 4444)
{"kind": "exporter", "data_type": "traces", "name": "debug"}
```
{{% /expand %}}

If everything worked as expected, you’re ready to continue building out the agent's YAML configuration file.
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: Adding the Hostmetric Receiver
linkTitle: 1.4 Hostmetric Receiver
weight: 4
hidden: true
---
### Setup

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ cd [WORKSHOP]/1-agent

Inside the `1-agent` directory, we will create a file containing the basic structure of an OpenTelemetry Collector configuration, defining the components you'll need to work with.

{{% expand title = "agent.yaml" %}}
To do this, create a file called `agent.yaml` and paste the following starting configuration into it:

```yaml
Expand All @@ -27,6 +28,10 @@ extensions:
endpoint: 0.0.0.0:13133

receivers:
hostmetrics: # Receiver Type
collection_interval: 3600s # Scrape metrics every hour
scrapers: # Array of hostmetric scrapers
cpu: # Scraper for cpu metrics

exporters:

Expand Down Expand Up @@ -58,6 +63,8 @@ service:
exporters: []
```
{{% /expand %}}
{{% tab title="Initial Directory Structure" %}}
```text
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
title: Configuring a File exporter
linkTitle: 2. File Exporter
weight: 2
---
### Different between Debug and FileExporter

To capture more than just debug output on the screen, we also want to generate output during the export phase of the pipeline. For this, we'll add a **File Exporter** to write OTLP data to files for comparison.

The difference between the OpenTelemetry **debug exporter** and the **file exporter** lies in their purpose and output destination:

| Feature | Debug Exporter | File Exporter |
|-----------------------|----------------------------------|--------------------------------|
| **Output Location** | Console/Log | File on disk |
| **Purpose** | Real-time debugging | Persistent offline analysis |
| **Best for** | Quick inspection during testing | Temporary storage and sharing |
| **Production Use** | No | Rare, but possible |
| **Persistence** | No | Yes |

In summary, the **debug exporter** is great for real-time, in-development troubleshooting, while the **file exporter** is better suited for storing telemetry data locally for later use.

Let's configure and add the `FileExporter`:

{{% notice title="Exercise" style="green" icon="running" %}}

- **Configuring a `file` exporter**: Add the following under the `exporters` section of your `agent.yaml`:

```yaml
file: # Exporter Type
path: "./agent.out" # Path where data will be saved in OTLP json format
append: false # Overwrite the file each time
```
- **Update the Pipelines Section**: Add the `file` exporter to the `metrics`, `traces` and `logs` pipelines (leave debug as the first in the array)

```yaml
#traces:
metrics: # Metrics Pipeline
receivers: [otlp] # Array of Metric Receivers
processors: # Array of Metric Processors
- memory_limiter # Handles memory limits for this Pipeline
exporters: [debug, file] # Array of Metric Exporters
#logs:
```

{{% /notice %}}
Original file line number Diff line number Diff line change
@@ -1,49 +1,8 @@
---
title: Configuring a File exporter
linkTitle: 1.2 Adding a File exporter
weight: 2
title: Testing the File exporter
linkTitle: 2.1 Testing the File Exporter
weight: 1
---
### Different between Debug and FileExporter

To capture more than just debug output on the screen, we also want to generate output during the export phase of the pipeline. For this, we'll add a **File Exporter** to write OTLP data to files for comparison.

The difference between the OpenTelemetry **debug exporter** and the **file exporter** lies in their purpose and output destination:

| Feature | Debug Exporter | File Exporter |
|-----------------------|----------------------------------|--------------------------------|
| **Output Location** | Console/Log | File on disk |
| **Purpose** | Real-time debugging | Persistent offline analysis |
| **Best for** | Quick inspection during testing | Temporary storage and sharing |
| **Production Use** | No | Rare, but possible |
| **Persistence** | No | Yes |

In summary, the **debug exporter** is great for real-time, in-development troubleshooting, while the **file exporter** is better suited for storing telemetry data locally for later use.

Let's configure and add the `FileExporter`:

{{% notice title="Exercise" style="green" icon="running" %}}

- **Configuring a `file` exporter**: Add the following under the `exporters` section of your `agent.yaml`:

```yaml
file: # Exporter Type
path: "./agent.out" # Path where data will be saved in OTLP json format
append: false # Overwrite the file each time
```
- **Update the Pipelines Section**: Add the `file` exporter to the `metrics`, `traces` and `logs` pipelines (leave debug as the first in the array)

```yaml
#traces:
metrics: # Metrics Pipeline
receivers: [otlp] # Array of Metric Receivers
processors: # Array of Metric Processors
- memory_limiter # Handles memory limits for this Pipeline
exporters: [debug, file] # Array of Metric Exporters
#logs:
```

{{% /notice %}}

### Validating your agent.yaml Configuration

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Adding Resource Metadata
linkTitle: 1.3 Resource Metadata
linkTitle: 3. Resource Metadata
weight: 3
---
### Setup
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Filelog Receiver
linkTitle: 3. Filelog Receiver
linkTitle: 4. Filelog Receiver
time: 10 minutes
weight: 3
weight: 4
---

The `FileLog` receiver in the OpenTelemetry Collector is used to ingest logs from files. It monitors specified files for new log entries and streams those logs into the Collector for further processing or exporting. It is useful for testing and development purposes.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Route data though a gateway
linkTitle: 2. Gateway Setup
linkTitle: 5. Gateway Setup
time: 10 minutes
weight: 2
weight: 5
---

### Initial Gateway Configuration
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Building Resilience in the OpenTelemetry Collector using the File Storage Extension
linkTitle: 4. Resilience
linkTitle: 6. Resilience
time: 10 minutes
weight: 4
weight: 6
---

We will walk through how to use OpenTelemetry Collector’s `file_storage` extension to build resilience into your telemetry pipeline. Specifically, we will demonstrate how to use the file storage extension for checkpointing, managing retries, and handling temporary failures effectively.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Using a Filter to Drop Spans in OpenTelemetry Collector
linkTitle: 5. Dropping Spans
linkTitle: 7. Dropping Spans
time: 10 minutes
weight: 5
weight: 7
---

In this section, we will explore how to use the **Filter Processor** to selectively drop spans based on certain conditions.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Removing Tags and Redacting Sensitive Data in OpenTelemetry Collector
linkTitle: 6. Sensitive Data
linkTitle: 8. Sensitive Data
time: 10 minutes
weight: 6
weight: 8
---

In this section, you'll learn how to configure the OpenTelemetry Collector to remove specific tags and redact sensitive data from your telemetry (traces). This is essential when dealing with sensitive information such as credit card numbers, personal data, or other security-related information that needs to be protected or anonymized.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Using the OpenTelemetry Transform Processor
linkTitle: 7. Transform Data
linkTitle: 9. Transform Data
time: 10 minutes
weight: 7
weight: 9
---

In this section, we will explore the **Transform Processor** in the OpenTelemetry Collector. This processor allows you to modify trace, metric, and log data at runtime by performing actions like adding, updating, or deleting attributes. It’s a powerful tool to reshape telemetry data before it is exported to the backend.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Routing
linkTitle: 8. Routing
linkTitle: 10. Routing
time: 10 minutes
weight: 8
weight: 10
---

The routing connector in OpenTelemetry is a powerful feature that allows you to direct data (`traces`, `metrics`, or l`ogs`) to different pipelines based on specific criteria. This is especially useful in scenarios where you want to apply different processing or exporting logic to subsets of your telemetry data.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
---
title: Wrap-up
linkTitle: 90. wrap-up
time: 1 minutes
weight: 9
linkTitle: 11. Wrap-up
weight: 11
---

![Well done](../images/welldone.png)

0 comments on commit 21d3ced

Please sign in to comment.