Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add beyla gce sample #105

Merged
merged 9 commits into from
Aug 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions recipes/beyla-golden-signals/gce/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# eBPF Golden Signals with Beyla on GCE

This recipe demonstrates how to configure the Google Cloud Ops Agent (installed on your Google Compute Engine instance) to produce golden signal metrics from [Beyla](https://github.com/grafana/beyla) http
metrics and send those metrics to [Google Managed Service for
Prometheus](https://cloud.google.com/stackdriver/docs/managed-prometheus).

In this recipe, Beyla is configured to generate http metrics from an application running on a GCE instance without introducing any code changes. The sample application used for this recipe is a simple Java server-client application located in the [sample-apps/java](../../../sample-apps/java/) directory of this repository.

This recipe is based on applying an [Google Cloud Ops Agent](https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent) configuration that enables the agent to receive and export telemetry generated by Beyla to [Google Managed Service for Prometheus](https://cloud.google.com/stackdriver/docs/managed-prometheus).

Beyla allows [two modes of exporting data](https://grafana.com/docs/beyla/latest/configure/export-modes/#beyla-export-modes). This sample relies on the *Direct Mode* which allows Beyla to push OTLP data to a specified endpoint.

# Prerequisites

* Cloud Monitoring API enabled in your GCP project
* The `roles/monitoring.metricWriter`
[IAM permissions](https://cloud.google.com/trace/docs/iam#roles) for your cluster's service
account (or Workload Identity setup as shown below).
* A GCE instance running Debian Linux and [Google Cloud Ops Agent](https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent) installed.
* For various installation methods see the [installation instructions](https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent/install-index).
* If you're creating the GCE instance from Google Cloud Console, you can simply check the `Install Ops Agent for Monitoring and Logging` checkbox during VM creation.
* Ability to SSH into the created GCE instance.
* The files in this recipe, locally on the created VM.
* You can `git clone` this repository directly on the VM and `cd` into this recipe's folder. *You may need to install `git` on the VM.*

## Running

From the location of this recipe *(beyla-golden-sigals/gce)*, execute the following:

### Configure the VM instance

#### Install and configure Beyla on the VM:

```sh
# Make sure to run this without a dot slash (./) since it needs to source variables.
# Running using ./ will run the script it in a subshell.
. configure-beyla.sh
```

#### Check if Google Cloud Ops Agent is still running (Optional):

```sh
# Logging Agent and Metrics Agent should be running
sudo systemctl status "google-cloud-ops-agent*"
```

#### Start Beyla process

> [!WARNING]
> This will start the Beyla process in the background as root.

```sh
# Start beyla in the background
psx95 marked this conversation as resolved.
Show resolved Hide resolved
sudo -E beyla &
```

## Build & run the sample Java server app

### Build the server and client app and start server in the background.

> [!TIP]
> If for whatever reason you need to kill the currently running server app, run the following command `fuser -k 8080/tcp`.

```sh
# This will start the built server application in the background, listening on port 8080.
./run-sample-app.sh
```

### Make requests running to the server from the client application.

```sh
# The sample app requires a URI to connect to
export SERVICE_NAME=http://localhost:8080
# Run the executable JAR for the app
java -jar ../../../sample-apps/java/app/build/libs/app-standalone.jar
```

Or, issue multiple requests to generate a good amount of metric data

```sh
export SERVICE_NAME=http://localhost:8080
# Issues 20 requests sequentially to the server
for i in {1..20}; do java -jar ../../../sample-apps/java/app/build/libs/app-standalone.jar; done;
```

## Viewing the generated metrics

To view the metrics generated by beyla:
- Metrics will be visible in the [Metrics Explorer](https://cloud.google.com/monitoring/charts/metrics-selector).
- The generated metrics will be located under *Prometheus Target* → *Http*.
Metrics to look for:
- `http_server_request_body_size`
- `http_server_request_duration`
54 changes: 54 additions & 0 deletions recipes/beyla-golden-signals/gce/configure-beyla.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
#!/bin/bash
#
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# This script installs and configures beyla on to the current GCE VM instance, installing necessary dependencies.
# The script also configures the installed google-cloud-ops-agent so that it can collect metrics emitted by Beyla.

# Update dependencies
sudo apt update

# Install Java
echo "Installing Java 17..."
sudo apt install -y openjdk-17-jdk

# Download Beyla
echo "Downloading Beyla..."
BEYLA_V1_7_RELEASE=https://github.com/grafana/beyla/releases/download/v1.7.0/beyla-linux-amd64-v1.7.0.tar.gz
curl -Lo beyla.tar.gz $BEYLA_V1_7_RELEASE
mkdir -p beyla-installation/
tar -xzf beyla.tar.gz -C beyla-installation/
# Move beyla executable to /usr/local/bin
# /usr/local/bin is the path on a default GCE instance running Debian
sudo cp beyla-installation/beyla /usr/local/bin

# Configuring Beyla Environment variables when running in direct mode
echo "Configuring Beyla..."
# Beyla configuration options
export BEYLA_OPEN_PORT=8080
export BEYLA_TRACE_PRINTER=text
export BEYLA_SERVICE_NAMESPACE="otel-beyla-gce-sample-service-ns"

# OpenTelemetry export settings
export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"
export OTEL_EXPORTER_OTLP_ENDPOINT="http://0.0.0.0:4317"
export OTEL_SERVICE_NAME="otel-beyla-gce-sample-service"
echo "Beyla Configured..."

# Apply the custom configuration to installed Google Cloud Ops Agent
echo "Configuring the Google Cloud Ops Agent..."
sudo cp ./google-cloud-ops-agent/config.yaml /etc/google-cloud-ops-agent/config.yaml
sudo systemctl restart google-cloud-ops-agent
echo "Google Cloud Ops Agent restarted..."
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# <== Enter custom agent configurations in this file.
# See https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent/configuration
# for more details.
#

combined:
receivers:
otlp:
type: otlp
grpc_endpoint: 0.0.0.0:4317
# Switch metrics mode to 'googlecloudmonitoring' to use Cloud Monitoring
# instead of Google managed service for prometheus
metrics_mode: googlemanagedprometheus
metrics:
service:
pipelines:
otlp:
receivers: [otlp]
# Add OTLP receiver to the following to export traces.
# The service for traces must exist when using 'combined' for configuration to be valid.
traces:
service:
pipelines:
otlp:
receivers: []
28 changes: 28 additions & 0 deletions recipes/beyla-golden-signals/gce/run-sample-app.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/bin/bash
#
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Build the server and client sample apps
SAMPLE_APP_JAVA_DIR=../../../sample-apps/java
SERVICE_JAR_FILE=service.jar

# Build client and server Java apps
pushd $SAMPLE_APP_JAVA_DIR
./gradlew :service:build
./gradlew :app:build
popd

# Run the server application
java -jar $SAMPLE_APP_JAVA_DIR/service/build/libs/$SERVICE_JAR_FILE &
Loading