Skip to content

Commit

Permalink
chore(external docs): improve vrl.dev tutorial (#21619)
Browse files Browse the repository at this point in the history
* chore(external docs): improve vrl.dev tutorial

* indent yaml

* fix condition

* fix indentation
  • Loading branch information
pront authored Oct 25, 2024
1 parent c169842 commit 85cfd0f
Showing 1 changed file with 51 additions and 28 deletions.
79 changes: 51 additions & 28 deletions website/content/en/docs/reference/vrl/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,15 @@ VRL programs act on a single observability [event](#event) and can be used to:
Those programs are specified as part of your Vector [configuration]. Here's an
example `remap` transform that contains a VRL program in the `source` field:

```toml {title="vector.toml"}
[transforms.modify]
type = "remap"
inputs = ["logs"]
source = '''
del(.user_info)
.timestamp = now()
'''
```YAML {title="vector.yaml"}
transforms:
modify:
type: remap
inputs:
- logs
source: |
del(.user_info)
.timestamp = now()
```
This program changes the contents of each event that passes through this
Expand All @@ -50,30 +51,49 @@ HTTP log events that look like this:
}
```

You want to apply these changes to each event:
Let's assume you want to apply a set of changes to each event that arrives to your Remap transform in order to produce
an event with the following fields:

- Parse the raw string into JSON
- Attempt to convert the timestamp and checks if the conversion was successful
- If the conversion is successful, convert the time to a UNIX timestamp; otherwise, use the current time
- Remove the `username` field
- Remove the temporary timestamp (`parsed_timestamp`) field
- Convert the `message` to lowercase
- `message` (string)
- `status` (int)
- `timestamp` (int)
- `timestamp_str` (timestamp)

This VRL program would accomplish all of that:
The following VRL program demonstrates how to achieve the above:

```coffee
# Parse the raw string into a JSON object, this way we can manipulate fields.
. = parse_json!(string!(.message))
.parsed_timestamp = parse_timestamp!(.timestamp, format: "%Y-%m-%dT%H:%M:%S.%fZ")

if is_timestamp(.parsed_timestamp) {
.timestamp = to_unix_timestamp(.parsed_timestamp)
# At this point `.` is the following:
#{
# "message": "SUCCESS",
# "status": 200,
# "timestamp": "2021-03-01T19:19:24.646170Z",
# "username": "ub40fan4life"
#}

# Attempt to parse the timestamp that was in the original message.
# Note that `.timestamp` can be `null` if it wasn't present.
parsed_timestamp, err = parse_timestamp(.timestamp, format: "%Y-%m-%dT%H:%M:%S.%fZ")

# Check if the conversion was successful. Note here that all errors must be handled, more on that later.
if err == null {
# Note that the `to_unix_timestamp` expects a `timestamp` argument.
# The following will compile because `parse_timestamp` returns a `timestamp`.
.timestamp = to_unix_timestamp(parsed_timestamp)
} else {
# Conversion failed, in this case use the current time.
.timestamp = to_unix_timestamp(now())
}

# Convert back to timestamp for this tutorial.
.timestamp_str = from_unix_timestamp!(.timestamp)

# Remove the `username` field from the final target.
del(.username)
del(.parsed_timestamp)

# Convert the `message` to lowercase.
.message = downcase(string!(.message))
```

Expand All @@ -83,7 +103,8 @@ Finally, the resulting event:
{
"message": "success",
"status": 200,
"timestamp": 1614626364
"timestamp": 1614644364,
"timestamp_str": "2021-03-02T00:19:24Z"
}
```

Expand All @@ -94,19 +115,21 @@ event. But you can also use VRL to specify conditions, which convert events into
a single Boolean expression. Here's an example [`filter`][filter] transform that
filters out all messages for which the `severity` field equals `"info"`:

```toml {title="vector.toml"}
[transforms.filter_out_info]
type = "filter"
inputs = ["logs"]
condition = '.severity != "info"'
```yaml {title="vector.yaml"}
transforms:
filter_out_info:
type: filter
inputs:
- logs
condition: '.severity != "info"'
```
Conditions can also be more multifaceted. This condition would filter out all
events for which the `severity` field is `"info"`, the `status_code` field is
greater than or equal to 400, and the `host` field isn't set:

```coffee
condition = '.severity != "info" && .status_code < 400 && exists(.host)
condition = '.severity != "info" && .status_code < 400 && exists(.host)'
```

{{< info title="More VRL examples" >}} You can find more VRL examples further
Expand Down

0 comments on commit 85cfd0f

Please sign in to comment.