Skip to content

Commit

Permalink
Manual merge due to GitHub outage
Browse files Browse the repository at this point in the history
  • Loading branch information
@rcastley
rcastley committed Jan 30, 2025
1 parent 21d3ced commit 906a846
Show file tree
Hide file tree
Showing 3 changed files with 170 additions and 90 deletions.
250 changes: 164 additions & 86 deletions content/en/ninja-workshops/10-advanced-otel/80-sensitive-data/_index.md
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: Removing Tags and Redacting Sensitive Data in OpenTelemetry Collector
linkTitle: 8. Sensitive Data
linkTitle: 6. Sensitive Data
time: 10 minutes
weight: 8
weight: 6
---

In this section, you'll learn how to configure the OpenTelemetry Collector to remove specific tags and redact sensitive data from your telemetry (traces). This is essential when dealing with sensitive information such as credit card numbers, personal data, or other security-related information that needs to be protected or anonymized.
Expand All @@ -22,101 +22,185 @@ Batch Processor: We'll use it to control how traces are batched and exported.
Memory Limiter: Ensures that the collector does not consume too much memory.
Redaction Processor: A dedicated processor for redacting sensitive data, including credit card numbers and account details.

### Step 1: Review the YAML Configuration
### Step 1: Initial Setup

Let's look at the YAML configuration for this workshop. This file contains various processors and how they are configured to remove and redact sensitive data.
On your machine, navigate to the directory where you're running the workshop. Create a new subdirectory called `6-remove-sensitive-data`, then copy the latest versions of `agent.yaml` and `trace.json` from `1-agent` into this new directory.

```yaml
processors:
# Batch processor to control batch size and metadata keys
batch:
metadata_keys:
- X-SF-Token

# Memory Limiter processor to prevent excessive memory usage
memory_limiter:
check_interval: 2s
limit_mib: 512

# Resource detection processor to detect system resource attributes
resourcedetection:
detectors: [system]
override: true

# Resource processor to add a new mode attribute
resource/add_mode:
attributes:
- action: insert
value: "agent"
key: otelcol.service.mode

# Attributes processor to remove specific tags (attributes) from traces
attributes/removetags:
actions:
- key: SPAN_TAG_KEY
action: delete

# Attributes processor to update or redact sensitive data
attributes/update:
actions:
- key: cc_number
value: redacted
action: update # Redacts credit card numbers with the word 'redacted'
- key: account_password
action: delete # Deletes account password attribute
- key: account_email
action: hash # Hashes the account email for privacy

# Redaction processor to apply custom redaction rules
redaction/update:
allow_all_keys: false # Block non-allowed keys
allowed_keys:
- description
- group
- id
- name
ignored_keys:
- safe_attribute # Safe attributes that won’t be redacted
blocked_values:
- "4[0-9]{12}(?:[0-9]{3})?" # Regular expression for Visa credit card numbers
- "(5[1-5][0-9]{14})" # Regular expression for MasterCard numbers
summary: debug # Diagnostic summary of redacted attributes
Next, move into the `[WORKSHOP]/6-remove-sensitive-data` directory.

```text
WORKSHOP
├── 1-agent
├── 2-gateway
├── 3-filelog
├── 4-resilience
├── 5-dropping-spans
├── 6-remove-sensitive-data
│ ├── agent.yaml
│ └── trace.json
└── otelcol
```

#### 1.2 Setup and Review Simulated Trace Data

For this section, you'll need to generate some trace data that includes sensitive data.

Copy the following JSON and save to the file named `trace.json` in the `6-remove-sensitive-data` directory:

{{% tabs %}}
{{% tab title="Compacted JSON" %}}

```json
{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"my.service"}},{"key":"deployment.environment","value":{"stringValue":"my.environment"}}]},"scopeSpans":[{"scope":{"name":"my.library","version":"1.0.0","attributes":[{"key":"my.scope.attribute","value":{"stringValue":"some scope attribute"}}]},"spans":[{"traceId":"5B8EFFF798038103D269B633813FC60C","spanId":"EEE19B7EC3C1B174","parentSpanId":"EEE19B7EC3C1B173","name":"I'm a server span","startTimeUnixNano":"1544712660000000000","endTimeUnixNano":"1544712661000000000","kind":2,"attributes":[{"key":"user.name","value":{"stringValue":"George Lucas"}},{"key":"user.user_id","value":{"stringValue":"oldbenkenobi"}},{"key":"user.phone_number","value":{"stringValue":"+1555-867-5309"}},{"key":"user.email","value":{"stringValue":"[email protected]"}},{"key":"user.account_password","value":{"stringValue":"LOTR>StarWars1-2-3"}},{"key":"user.visa","value":{"stringValue":"4111 1111 1111 1111"}},{"key":"user.amex","value":{"stringValue":"3782 822463 10005"}},{"key":"user.mastercard","value":{"stringValue":"5555 5555 5555 4444"}}]}]}]}]}
```

{{% /tab %}}

{{% tab title="Formatted JSON" %}}

```json
{
"resourceSpans": [
{
"resource": {
"attributes": [
{
"key": "service.name",
"value": {
"stringValue": "my.service"
}
},
{
"key": "deployment.environment",
"value": {
"stringValue": "my.environment"
}
}
]
},
"scopeSpans": [
{
"scope": {
"name": "my.library",
"version": "1.0.0",
"attributes": [
{
"key": "my.scope.attribute",
"value": {
"stringValue": "some scope attribute"
}
}
]
},
"spans": [
{
"traceId": "5B8EFFF798038103D269B633813FC60C",
"spanId": "EEE19B7EC3C1B174",
"parentSpanId": "EEE19B7EC3C1B173",
"name": "I'm a server span",
"startTimeUnixNano": "1544712660000000000",
"endTimeUnixNano": "1544712661000000000",
"kind": 2,
"attributes": [
{
"key": "user.name",
"value": {
"stringValue": "George Lucas"
}
},
{
"key": "user.user_id",
"value": {
"stringValue": "oldbenkenobi"
}
},
{
"key": "user.phone_number",
"value": {
"stringValue": "+1555-867-5309"
}
},
{
"key": "user.email",
"value": {
"stringValue": "[email protected]"
}
},
{
"key": "user.account_password",
"value": {
"stringValue": "LOTR>StarWars1-2-3"
}
},
{
"key": "user.visa",
"value": {
"stringValue": "4111 1111 1111 1111"
}
},
{
"key": "user.amex",
"value": {
"stringValue": "3782 822463 10005"
}
},
{
"key": "user.mastercard",
"value": {
"stringValue": "5555 5555 5555 4444"
}
}
]
}
]
}
]
}
]
}
```

{{% /tab %}}
{{% /tabs %}}

In this exercise, we will update the `agent.yaml` file by adding `attribute` and `redaction` processors.

### Step 2 Configure Agent

### Breakdown of Key Configurations

#### 1. Remove Tags (Attributes)
#### 2.1 Remove Tags (Attributes)

The `attributes/removetags` processor allows you to delete specific attributes (tags) from spans. In this case, we're removing the tag `SPAN_TAG_KEY`:
The `attributes/removetags` processor allows you to delete specific attributes (tags) from spans. In this case, we're removing the tag `user.user_id`:

```yaml
attributes/removetags:
actions:
- key: SPAN_TAG_KEY
- key: user.user_id
action: delete
```
#### 2. Update Sensitive Data
#### 2.2. Update Sensitive Data
The attributes/update processor is used to update or redact sensitive data. We perform the following actions:
Redacting credit card numbers: Replace the cc_number with the word "redacted".
Redacting credit card numbers: Replace the amex card number with the word "redacted".
Deleting the account_password field to remove passwords from traces.
Hashing the account_email to anonymize email addresses.
```yaml
attributes/update:
actions:
- key: cc_number
- key: user.amex
value: redacted
action: update
- key: account_password
- key: user.account_password
action: delete
- key: account_email
- key: user.account_email
action: hash
```
#### 3. Redaction Processor
#### 2.3. Redaction Processor
The redaction/update processor provides fine-grained control over which attributes are allowed or blocked from traces. We configure this processor to:
Expand All @@ -139,35 +223,29 @@ redaction/update:
summary: debug # Show detailed debug information about redactions
```

### Step 2: Apply and Test the Configuration

#### 2.1 Save the Configuration
Open the `agent.yaml` and add the `attributes/removetags`, `attributes/update`, and `redaction/update` configuration to the `processors` section:

Save the YAML configuration above into a file called otel-collector-config.yaml.
{{% notice title="Exercise" style="green" icon="running" %}}

#### 2.2 Run the OpenTelemetry Collector
- **Configure the processors**:
Open the `agent.yaml` and add the `attributes/removetags`, `attributes/update`, and `redaction/update` configuration to the `processors` section

To run the OpenTelemetry Collector with your configuration file, use the following command:

```bash
otelcol --config=otel-collector-config.yaml
```
- **Add the `attribute` and `redaction` processors**: Make sure you add the processors to the `traces` pipeline.

### 2.3 Generate Some Sample Data
{{% /notice %}}

To test the configuration, generate some sample telemetry data (traces) that includes sensitive attributes such as:
Validate the agent configuration using **[otelbin.io](https://www.otelbin.io/)**, the results for the `traces` pipeline should look like this:

- cc_number (credit card number),
- account_password (password),
- account_email (email address).
![otelbin-f-6-1-traces](../images/otelbin-f-6-1-trace.png)

You can use a test application or OpenTelemetry instrumentation to simulate these traces.
### Step 2: Verify the Redaction

### 2.4 Verify the Redaction
Ensure that the `agent` is started the `[WORKSHOP]/6-sensitive-data` folder using the correct agent configuration yaml. Next, update and use the **cURL** command we used earlier to send the `health.json` payload, and send the `trace.json` data created above.

After the collector processes the data, verify that:

- The cc_number field is redacted with the word redacted.
- The user_id field is deleted.
- The amex cc number field is redacted with the word redacted.
- The account_password field is deleted.
- The account_email field is hashed.
- Sensitive credit card numbers (Visa, MasterCard) are properly masked using regex.
Expand Down
Binary file added content/en/ninja-workshops/10-advanced-otel/images/otelbin-f-6-1-trace.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 6 additions & 4 deletions workshop/ninja/advanced-otel/5-removing-tags-redacting-sensitive-data/agent-5.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,18 +66,18 @@ processors:
# Overrides an existing tag for a span.
attributes/removetags:
actions:
- key: SPAN_TAG_KEY
- key: user.user_id
action: delete

# Updates attribute values to remove and redact sensitive data
attributes/update:
actions:
- key: cc_number
- key: user.amex
value: redacted
action: update
- key: account_password
- key: user.account_password
action: delete
- key: account_email
- key: user.account_email
action: hash

# Redaction processor
Expand Down Expand Up @@ -125,6 +125,8 @@ service:
- resourcedetection
- resource/add_mode
- attributes/removetags
- attributes/update
- redaction/update
exporters: [otlp/gateway, debug]
metrics:
receivers: [otlp]
Expand Down

0 comments on commit 906a846

Please sign in to comment.