Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: exporter config #214

Merged
merged 3 commits into from
Nov 7, 2024
+67 −47
Merged

refactor: exporter config #214

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
aaf4f32
refactor: exporter config
lvlcn-t Nov 5, 2024
7ff5003
chore: simplified if guard
puffitos Nov 6, 2024
cfbfdf1
docs: add certpath hint
lvlcn-t Nov 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 21 additions & 14 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,15 +81,14 @@ The binary is available for several distributions. To install the binary, use a
Replace `${RELEASE_VERSION}` with the desired release version:

```sh
curl https://github.com/caas-team/sparrow/releases/download/v${RELEASE_VERSION}/sparrow_${RELEASE_VERSION}_linux_amd64.tar.gz -Lo sparrow.tar.gz
curl https://github.com/caas-team/sparrow/releases/download/v${RELEASE_VERSION}/sparrow_${RELEASE_VERSION}_checksums.txt -Lo checksums.txt
export RELEASE_VERSION=0.5.0
```

For example, for release `v0.3.1`:
Download the binary:

```sh
curl https://github.com/caas-team/sparrow/releases/download/v0.3.1/sparrow_0.3.1_linux_amd64.tar.gz -Lo sparrow.tar.gz
curl https://github.com/caas-team/sparrow/releases/download/v0.3.1/sparrow_0.3.1_checksums.txt -Lo checksums.txt
curl https://github.com/caas-team/sparrow/releases/download/v${RELEASE_VERSION}/sparrow_${RELEASE_VERSION}_linux_amd64.tar.gz -Lo sparrow.tar.gz
curl https://github.com/caas-team/sparrow/releases/download/v${RELEASE_VERSION}/sparrow_${RELEASE_VERSION}_checksums.txt -Lo checksums.txt
```

Extract the binary:
Expand Down Expand Up @@ -291,9 +290,13 @@ telemetry:
# The token to use for authentication.
# If the exporter does not require a token, this can be left empty.
token: ""
# The path to the tls certificate to use.
# To disable tls, either set this to an empty string or set it to insecure.
certPath: ""
# Configures tls for the telemetry exporter
tls:
# Enable or disable TLS
enabled: true
# The path to the tls certificate to use.
# Only required if your otel endpoint uses custom TLS certificates
certPath: ""
```

#### Loader
Expand Down Expand Up @@ -644,17 +647,21 @@ Replace `<sparrow_instance_address>` with the actual address of your `sparrow` i

The `sparrow` supports exporting telemetry data using the OpenTelemetry Protocol (OTLP). This allows users to choose their preferred telemetry provider and collector. The following configuration options are available for setting up telemetry:

| Field | Type | Description |
| ---------- | -------- | ------------------------------------------------------------------------ |
| `exporter` | `string` | The telemetry exporter to use. Options: `grpc`, `http`, `stdout`, `noop` |
| `url` | `string` | The address to export telemetry to |
| `token` | `string` | The token to use for authentication |
| `certPath` | `string` | The path to the TLS certificate to use |
| Field | Type | Description |
| -------------- | -------- | --------------------------------------------------------------------------- |
| `enabled` | `bool` | Whether to enable telemetry. Default: `false` |
| `exporter` | `string` | The telemetry exporter to use. Options: `grpc`, `http`, `stdout`, `noop` |
| `url` | `string` | The address to export telemetry to. |
| `token` | `string` | The token to use for authentication. |
| `tls.enabled` | `bool` | Enable or disable TLS. |
| `tls.certPath` | `string` | The path to the TLS certificate to use. Only required if custom TLS is used |

For example, to export telemetry data using OTLP via gRPC, you can add the following configuration to your [startup configuration](#startup):

```yaml
telemetry:
# Whether to enable telemetry. (default: false)
enabled: true
# The telemetry exporter to use.
# Options:
# grpc: Exports telemetry using OTLP via gRPC.
Expand Down
19 changes: 10 additions & 9 deletions pkg/sparrow/metrics/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,17 @@ type Config struct {
// Url is the Url of the collector to which the traces are exported
Url string `yaml:"url" mapstructure:"url"`
// Token is the token used to authenticate with the collector
Token string `yaml:"token" mapstructure:"token"`
Tls TLSConfig `yaml:"tls" mapstructure:"tls"`
Token string `yaml:"token" mapstructure:"token"`
// TLS holds the tls configuration
TLS TLSConfig `yaml:"tls" mapstructure:"tls"`
}

type TLSConfig struct {
// CertPath is the path to the tls certificate file
// Enabled is a flag to enable or disable the tls
Enabled bool `yaml:"enabled" mapstructure:"enabled"`
// CertPath is the path to the tls certificate file.
// This is only required if the otel backend uses custom TLS certificates.
CertPath string `yaml:"certPath" mapstructure:"certPath"`
Enabled bool `yaml:"enabled" mapstructure:"enabled"`
}

func (c *Config) Validate(ctx context.Context) error {
Expand All @@ -51,11 +54,9 @@ func (c *Config) Validate(ctx context.Context) error {
return err
}

if c.Exporter.IsExporting() {
if c.Url == "" {
log.ErrorContext(ctx, "Url is required for otlp exporter", "exporter", c.Exporter)
return fmt.Errorf("url is required for otlp exporter %q", c.Exporter)
}
if c.Exporter.IsExporting() && c.Url == "" {
log.ErrorContext(ctx, "Url is required for otlp exporter", "exporter", c.Exporter)
return fmt.Errorf("url is required for otlp exporter %q", c.Exporter)
}
return nil
}
60 changes: 36 additions & 24 deletions pkg/sparrow/metrics/exporter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,44 +91,44 @@ func (e Exporter) Create(ctx context.Context, config *Config) (sdktrace.SpanExpo

// newHTTPExporter creates a new HTTP exporter
func newHTTPExporter(ctx context.Context, config *Config) (sdktrace.SpanExporter, error) {
headers, tlsCfg, err := getCommonConfig(config)
cfg, err := newExporterConfig(config)
if err != nil {
return nil, err
}

opts := []otlptracehttp.Option{
otlptracehttp.WithEndpoint(config.Url),
otlptracehttp.WithHeaders(headers),
otlptracehttp.WithHeaders(cfg.headers),
}
if config.Tls.Enabled {
if tlsCfg != nil {
opts = append(opts, otlptracehttp.WithTLSClientConfig(tlsCfg))
}
} else {
if !config.TLS.Enabled {
opts = append(opts, otlptracehttp.WithInsecure())
return otlptracehttp.New(ctx, opts...)
}
if cfg.tls != nil {
opts = append(opts, otlptracehttp.WithTLSClientConfig(cfg.tls))
}

return otlptracehttp.New(ctx, opts...)
}

// newGRPCExporter creates a new gRPC exporter
func newGRPCExporter(ctx context.Context, config *Config) (sdktrace.SpanExporter, error) {
headers, tlsCfg, err := getCommonConfig(config)
cfg, err := newExporterConfig(config)
if err != nil {
return nil, err
}

opts := []otlptracegrpc.Option{
otlptracegrpc.WithEndpoint(config.Url),
otlptracegrpc.WithHeaders(headers),
otlptracegrpc.WithHeaders(cfg.headers),
}

if !config.Tls.Enabled {
if !config.TLS.Enabled {
opts = append(opts, otlptracegrpc.WithInsecure())
return otlptracegrpc.New(ctx, opts...)
}
if tlsCfg != nil {
opts = append(opts, otlptracegrpc.WithTLSCredentials(credentials.NewTLS(tlsCfg)))
if cfg.tls != nil {
opts = append(opts, otlptracegrpc.WithTLSCredentials(credentials.NewTLS(cfg.tls)))
}

return otlptracegrpc.New(ctx, opts...)
Expand All @@ -144,22 +144,36 @@ func newNoopExporter(_ context.Context, _ *Config) (sdktrace.SpanExporter, error
return nil, nil
}

// getCommonConfig returns the common configuration for the exporters
func getCommonConfig(config *Config) (map[string]string, *tls.Config, error) {
headers := make(map[string]string)
// exporterConfig contains the common configuration for the exporters
type exporterConfig struct {
// headers contains the headers to send with spans
headers map[string]string
// tls contains the TLS configuration for the exporter
tls *tls.Config
}

// newExporterConfig returns the common configuration for the exporters
func newExporterConfig(config *Config) (exporterConfig, error) {
headers := map[string]string{}
if config.Token != "" {
headers["Authorization"] = fmt.Sprintf("Bearer %s", config.Token)
}

if config.Tls.Enabled {
tlsCfg, err := getTLSConfig(config.Tls.CertPath)
if config.TLS.Enabled {
tlsCfg, err := getTLSConfig(config.TLS.CertPath)
if err != nil {
return nil, nil, fmt.Errorf("failed to create TLS configuration: %w", err)
return exporterConfig{}, fmt.Errorf("failed to create TLS configuration: %w", err)
}
return headers, tlsCfg, nil
return exporterConfig{
headers: headers,
tls: tlsCfg,
}, nil
}

return headers, nil, nil
return exporterConfig{
headers: headers,
tls: nil,
}, nil
}

// FileOpener is the function used to open a file
Expand All @@ -172,7 +186,7 @@ var openFile FileOpener = func() FileOpener {
}
}()

func getTLSConfig(certFile string) (conf *tls.Config, err error) {
func getTLSConfig(certFile string) (*tls.Config, error) {
if certFile == "" {
return nil, nil
}
Expand All @@ -182,9 +196,7 @@ func getTLSConfig(certFile string) (conf *tls.Config, err error) {
return nil, fmt.Errorf("failed to open certificate file: %w", err)
}
defer func() {
if cErr := file.Close(); cErr != nil {
err = errors.Join(err, cErr)
}
err = errors.Join(err, file.Close())
}()

b, err := io.ReadAll(file)
Expand Down
Loading