Skip to content

Commit

Permalink
initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
@lohbe
lohbe committed Apr 21, 2020
0 parents commit bb48726
Show file tree
Hide file tree
Showing 10 changed files with 1,199 additions and 0 deletions.
43 changes: 43 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Wavefront Proxy Kustomization

This repository consolidates advanced configuration of [Wavefront proxy](https://docs.wavefront.com/proxies.html) through [Kustomize](https://kustomize.io). This allows easy reference of configuration parameters and deployment of a customised Wavefront proxy in Kubernetes. This is a good starting point to modify Wavefront proxy deployment to existing automation pipelines, especially if not using the Wavefront [Helm chart](https://docs.wavefront.com/kubernetes.html#kubernetes-quick-install-using-helm).

Source: Wavefront Proxy Advanced Configuration [Guide](https://docs.wavefront.com/proxies_configuring.html)

## Getting Started

This guide assumes that you are familiar with Kustomize and have a ready cluster (e.g. minikube) with `kubectl` to run/test the kustomizations.

* Install [Kustomize](https://github.com/kubernetes-sigs/kustomize/blob/master/docs/INSTALL.md) on your machine
* Clone this repository and note the structure as follows:

`base/` - adds [`wavefront.conf`](https://github.com/wavefrontHQ/wavefront-proxy/blob/master/pkg/etc/wavefront/wavefront-proxy/wavefront.conf.default) and [`log4j.xml`](https://github.com/wavefrontHQ/wavefront-proxy/blob/master/pkg/etc/wavefront/wavefront-proxy/log4j2.xml.default) advanced configuration to upstream K8s proxy [manifest](https://raw.githubusercontent.com/wavefrontHQ/wavefront-kubernetes/master/wavefront-proxy/wavefront.yaml). Environment variables override configuration files.

`overlays/memory-limit/` - overlays, or "inherits" `base/` and limits memory use of Wavefront proxy by adjusting Java heap size and patching the K8s [container](https://docs.wavefront.com/proxies_configuring.html#configuring-a-proxy-in-a-container) resource limits.

* Add your own `WAVEFRONT_TOKEN` and `WAVEFRONT_URL` credentials to `base/kustomization.yaml`

* To preview the generated manifest, run in `base/` or `memory-limit/` directory:

`$ kustomize build`

* To apply the generated manifest:

`$ kustomize build | kubectl apply --namespace wavefront -f -`

Alternatively, you may use built-in support for Kustomize: `kubectl apply -k .`

* To cleanup and remove the applied manifests, replace `apply` with `delete`

### Note about secrets management

This kustomization does not integrate K8s secrets management. For non-trivial deployments, it is recommended to keep your Wavefront tokens safe. For more details, please visit [Vault](https://github.com/hashicorp/vault) or [SealedSecrets](https://github.com/bitnami-labs/sealed-secrets), depending on your environment and security needs.

## Future Improvements

- Add native remote loading of upstream resources when this feature stabilises in Kustomize. [1], [2]
- Add [preprocessor](https://github.com/wavefrontHQ/wavefront-proxy/blob/master/pkg/etc/wavefront/wavefront-proxy/preprocessor_rules.yaml.default) rules configuration

[1]: https://github.com/kubernetes-sigs/kustomize/pull/2167
[2]: https://github.com/kubernetes-sigs/kustomize/issues/970

146 changes: 146 additions & 0 deletions base/configs/log4j2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO" monitorInterval="5">
<Properties>
<Property name="log-path">/var/log/wavefront</Property>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout>
<!-- Use the pattern below to output log in the same format as older versions
<pattern>%d{MMM d, yyyy h:mm:ss a} %C{10} %M%n%p{WARN=WARNING, DEBUG=FINE, ERROR=SEVERE}: %m%n</pattern>
-->
<pattern>%d %-5level [%c{1}:%M] %m%n</pattern>
</PatternLayout>
</Console>
<RollingFile name="RollingFile" fileName="${log-path}/wavefront.log"
filePattern="${log-path}/wavefront-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<!-- Use the pattern below to output log in the same format as older versions
<pattern>%d{MMM d, yyyy h:mm:ss a} %C{10} %M%n%p{WARN=WARNING, DEBUG=FINE, ERROR=SEVERE}: %m%n</pattern>
-->
<pattern>%d %-5level [%c{1}:%M] %m%n</pattern>
</PatternLayout>

<!-- Log rollover and retention policies. Default settings: roll over logs once a day or every 100 MB,
whichever comes first; keep max 10 log files per day for no more than 31 days -->
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="100 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-2*.log" />
<IfLastModified age="31d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
<!-- Uncomment the RollingFile section below to log blocked points to a file -->
<!--
<RollingFile name="BlockedPointsFile" fileName="${log-path}/wavefront-blocked-points.log"
filePattern="${log-path}/wavefront-blocked-points-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<pattern>%m%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="100 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-blocked*.log" />
<IfLastModified age="31d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
-->
<!-- Uncomment the RollingFile section below to log all valid points to a file -->
<!--
<RollingFile name="ValidPointsFile" fileName="${log-path}/wavefront-valid-points.log"
filePattern="${log-path}/wavefront-valid-points-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<pattern>%m%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="1024 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-valid*.log" />
<IfLastModified age="7d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
-->
<!-- Uncomment the RollingFile section below to log all Jaeger tracing spans to a file -->
<!--
<RollingFile name="JaegerDataFile" fileName="${log-path}/wavefront-jaeger-data.log"
filePattern="${log-path}/wavefront-jaeger-data-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<pattern>%m%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="1024 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-jaeger*.log" />
<IfLastModified age="7d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
-->
<!-- Uncomment the RollingFile section below to log all Zipkin tracing spans to a file -->
<!--
<RollingFile name="ZipkinDataFile" fileName="${log-path}/wavefront-zipkin-data.log"
filePattern="${log-path}/wavefront-zipkin-data-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<pattern>%m%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="1024 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-zipkin*.log" />
<IfLastModified age="7d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
-->
</Appenders>
<Loggers>
<!-- Uncomment AppenderRef to log blocked points to a file.
Logger property level="WARN" logs only rejected points, level="INFO"
logs points filtered out by whitelist/blacklist rules as well -->
<AsyncLogger name="RawBlockedPoints" level="WARN" additivity="false">
<!--
<AppenderRef ref="BlockedPointsFile"/>
-->
</AsyncLogger>
<!-- Uncomment AppenderRef and set level="ALL" to log all valid points to a file -->
<AsyncLogger name="RawValidPoints" level="DEBUG" additivity="false">
<!--
<AppenderRef ref="ValidPointsFile"/>
-->
</AsyncLogger>
<!-- Uncomment AppenderRef and set level="ALL" to log Jaeger data to a file -->
<AsyncLogger name="JaegerDataLogger" level="DEBUG" additivity="false">
<!--
<AppenderRef ref="JaegerDataFile"/>
-->
</AsyncLogger>
<!-- Uncomment AppenderRef and set level="ALL" to log Zipkin data to a file -->
<AsyncLogger name="ZipkinDataLogger" level="DEBUG" additivity="false">
<!--
<AppenderRef ref="ZipkinDataFile"/>
-->
</AsyncLogger>
<Root level="INFO">
<AppenderRef ref="RollingFile"/>
<AppenderRef ref="Console" />
</Root>
</Loggers>
</Configuration>
146 changes: 146 additions & 0 deletions base/configs/log4j2.xml.default
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO" monitorInterval="5">
<Properties>
<Property name="log-path">/var/log/wavefront</Property>
</Properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout>
<!-- Use the pattern below to output log in the same format as older versions
<pattern>%d{MMM d, yyyy h:mm:ss a} %C{10} %M%n%p{WARN=WARNING, DEBUG=FINE, ERROR=SEVERE}: %m%n</pattern>
-->
<pattern>%d %-5level [%c{1}:%M] %m%n</pattern>
</PatternLayout>
</Console>
<RollingFile name="RollingFile" fileName="${log-path}/wavefront.log"
filePattern="${log-path}/wavefront-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<!-- Use the pattern below to output log in the same format as older versions
<pattern>%d{MMM d, yyyy h:mm:ss a} %C{10} %M%n%p{WARN=WARNING, DEBUG=FINE, ERROR=SEVERE}: %m%n</pattern>
-->
<pattern>%d %-5level [%c{1}:%M] %m%n</pattern>
</PatternLayout>

<!-- Log rollover and retention policies. Default settings: roll over logs once a day or every 100 MB,
whichever comes first; keep max 10 log files per day for no more than 31 days -->
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="100 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-2*.log" />
<IfLastModified age="31d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
<!-- Uncomment the RollingFile section below to log blocked points to a file -->
<!--
<RollingFile name="BlockedPointsFile" fileName="${log-path}/wavefront-blocked-points.log"
filePattern="${log-path}/wavefront-blocked-points-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<pattern>%m%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="100 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-blocked*.log" />
<IfLastModified age="31d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
-->
<!-- Uncomment the RollingFile section below to log all valid points to a file -->
<!--
<RollingFile name="ValidPointsFile" fileName="${log-path}/wavefront-valid-points.log"
filePattern="${log-path}/wavefront-valid-points-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<pattern>%m%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="1024 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-valid*.log" />
<IfLastModified age="7d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
-->
<!-- Uncomment the RollingFile section below to log all Jaeger tracing spans to a file -->
<!--
<RollingFile name="JaegerDataFile" fileName="${log-path}/wavefront-jaeger-data.log"
filePattern="${log-path}/wavefront-jaeger-data-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<pattern>%m%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="1024 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-jaeger*.log" />
<IfLastModified age="7d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
-->
<!-- Uncomment the RollingFile section below to log all Zipkin tracing spans to a file -->
<!--
<RollingFile name="ZipkinDataFile" fileName="${log-path}/wavefront-zipkin-data.log"
filePattern="${log-path}/wavefront-zipkin-data-%d{yyyy-MM-dd}-%i.log" >
<PatternLayout>
<pattern>%m%n</pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="1024 MB"/>
</Policies>
<DefaultRolloverStrategy max="10">
<Delete basePath="${log-path}" maxDepth="1">
<IfFileName glob="wavefront-zipkin*.log" />
<IfLastModified age="7d" />
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
-->
</Appenders>
<Loggers>
<!-- Uncomment AppenderRef to log blocked points to a file.
Logger property level="WARN" logs only rejected points, level="INFO"
logs points filtered out by whitelist/blacklist rules as well -->
<AsyncLogger name="RawBlockedPoints" level="WARN" additivity="false">
<!--
<AppenderRef ref="BlockedPointsFile"/>
-->
</AsyncLogger>
<!-- Uncomment AppenderRef and set level="ALL" to log all valid points to a file -->
<AsyncLogger name="RawValidPoints" level="DEBUG" additivity="false">
<!--
<AppenderRef ref="ValidPointsFile"/>
-->
</AsyncLogger>
<!-- Uncomment AppenderRef and set level="ALL" to log Jaeger data to a file -->
<AsyncLogger name="JaegerDataLogger" level="DEBUG" additivity="false">
<!--
<AppenderRef ref="JaegerDataFile"/>
-->
</AsyncLogger>
<!-- Uncomment AppenderRef and set level="ALL" to log Zipkin data to a file -->
<AsyncLogger name="ZipkinDataLogger" level="DEBUG" additivity="false">
<!--
<AppenderRef ref="ZipkinDataFile"/>
-->
</AsyncLogger>
<Root level="INFO">
<AppenderRef ref="RollingFile"/>
<AppenderRef ref="Console" />
</Root>
</Loggers>
</Configuration>
Loading

0 comments on commit bb48726

Please sign in to comment.