Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NMS-16149: Added confd to Sentinel #6820

Merged
merged 6 commits into from
Nov 10, 2023
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions opennms-container/sentinel/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
sentinel-config-schema.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can drop this one as well, -1 line 😁

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

/tarball-root
77 changes: 77 additions & 0 deletions opennms-container/sentinel/CONFD_README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Configuring Sentinel via confd
(instructions for testing/developing confd templates are given at the end of this document)
## Mounting
When starting the Sentinel container, mount a yaml file to the following path `/opt/sentinel/sentinel-config.yaml`.

Any configuration provided to confd will overwrite configuration specified as environment variables. Direct overlay of
specific configuration files will overwrite the corresponding config provided by confd.

## Contents
The following describes the keys that can be specified in `sentinel-config.yaml` to configure the Sentinel via confd.

### Sentinel Controller Config
```yaml
---
broker-url: "<broker url>"
id: "<id>"
location: "<location>"
```
Config specified will be written to `etc/org.opennms.sentinel.controller.cfg`.

#### User/Password
Supplying the broker username/password via yaml file for configuration via confd is not supported.

### Sentinel Elasticsearch Config
```yaml
---
elasticsearch:
url: "http://elastic-ip:9200"
index-strategy: "hourly"
replicas: 0
conn-timeout: 30000
read-timeout: 60000
```
Config specified will be written to `etc/org.opennms.features.flows.persistence.elastic.cfg`.

### Sentinel Datasource Config
```yaml
---
datasource:
url: "jdbc:postgresql://localhost:5432/opennms"
username: "postgres"
password: "postgres"
database-name: "opennms"
```
Config specified will be written to `etc/org.opennms.netmgt.distributed.datasource.cfg`.

### Sentinel Kafka Config
```yaml
---
ipc:
kafka:
bootstrap.servers: "my-kafka-ip-1:9092,my-kafka-ip-2:9092"
group.id: "OpenNMS"
```
Config specified will be written to `etc/org.opennms.core.ipc.sink.kafka.cfg` and `etc/org.opennms.core.ipc.sink.kafka.consumer.cfg`.

### Telemetry Flow Adapters
Individual flow adapters can be configured. See the example below for how to specify parameters and parsers. Any number
of uniquely named listeners can be defined.
```yaml
---
telemetry:
flows:
adapters:
NetFlow-5:
class-name: "org.opennms.netmgt.telemetry.protocols.netflow.adapter.netflow5.Netflow5Adapter"
parameters:
some-key: "some-value"
```
Config specified will be written to `deploy/confd-flows-feature.xml`.

### Instance Id
```yaml
---
org.opennms.instance.id: "<instance id>"
```
Config specified will be written to `etc/custom.system.properties`.
10 changes: 9 additions & 1 deletion opennms-container/sentinel/Dockerfile
Original file line number Diff line number Diff line change
@@ -98,13 +98,21 @@ RUN install -d -o 10001 -g 0 -m 775 \
ln -s /usr/share/sentinel/etc /etc/sentinel && \
ln -s /usr/share/sentinel /opt/sentinel

COPY ./container-fs/* /
# Install entrypoint wrapper and health check script
COPY container-fs/entrypoint.sh /
COPY container-fs/health.sh /

# Allow to send ICMP messages as non-root user
RUN setcap cap_net_raw+ep ${JAVA_HOME}/bin/java && \
echo ${JAVA_HOME}/lib/jli > /etc/ld.so.conf.d/java-latest.conf && \
ldconfig

# Install confd.io configuration files and scripts and ensure they are executable
COPY ./container-fs/confd/ /opt/sentinel/confd/
RUN chmod +x /opt/sentinel/confd/scripts/*
COPY ./sentinel-config-schema.yml /opt/sentinel/confd/
RUN touch /opt/sentinel/sentinel-config.yaml

VOLUME [ "/usr/share/sentinel/deploy", "/usr/share/sentinel/etc", "/usr/share/sentinel/data" ]

WORKDIR /usr/share/sentinel
1 change: 1 addition & 0 deletions opennms-container/sentinel/Makefile
Original file line number Diff line number Diff line change
@@ -6,5 +6,6 @@ CONTAINER_TYPE_FRIENDLY := Sentinel
DOCKER_PROJECT := sentinel
TARBALL := ../../opennms-assemblies/sentinel/target/*sentinel*.tar.gz
TAR_STRIP_COMPONENTS := 1
ADDITIONAL_TARGETS := sentinel-config-schema.yml

include ../common.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[template]
src = "confd-flows-feature.xml.tmpl"
dest = "/opt/sentinel/deploy/confd-flows-feature.xml"
keys = [
"/telemetry/flows/adapters"
]

reload_cmd = "/opt/sentinel/confd/scripts/remove-if-empty /opt/sentinel/deploy/confd-flows-feature.xml"
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[template]
src = "custom.system.properties.tmpl"
dest = "/opt/sentinel/etc/custom.system.properties"
keys = [
"/org.opennms.instance.id",
]
reload_cmd = "/opt/sentinel/confd/scripts/remove-if-empty /opt/sentinel/etc/custom.system.properties"
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[template]
src = "ipc-strategy.boot.tmpl"
dest = "/opt/sentinel/etc/featuresBoot.d/ipc-strategy.boot"
keys = [
"/ipc/kafka/bootstrap-servers"
]
reload_cmd = "/opt/sentinel/confd/scripts/remove-if-empty /opt/sentinel/etc/featuresBoot.d/ipc-strategy.boot"
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[template]
src = "org.opennms.core.ipc.sink.kafka.cfg.tmpl"
dest = "/opt/sentinel/etc/org.opennms.core.ipc.sink.kafka.cfg"
keys = [
"/ipc/kafka"
]
reload_cmd = "/opt/sentinel/confd/scripts/remove-if-empty /opt/sentinel/etc/org.opennms.core.ipc.sink.kafka.cfg"
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
[template]
src = "org.opennms.core.ipc.sink.kafka.consumer.cfg.tmpl"
dest = "/opt/sentinel/etc/org.opennms.core.ipc.sink.kafka.consumer.cfg"
keys = [
"/ipc/kafka"
]
reload_cmd = "/opt/sentinel/confd/scripts/remove-if-empty /opt/sentinel/etc/org.opennms.core.ipc.sink.kafka.consumer.cfg"
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[template]
src = "org.opennms.features.flows.persistence.elastic.cfg.tmpl"
dest = "/opt/sentinel/etc/org.opennms.features.flows.persistence.elastic.cfg"
keys = [
"/elasticsearch/url",
"/elasticsearch/index-strategy",
"/elasticsearch/replicas",
"/elasticsearch/conn-timeout",
"/elasticsearch/read-timeout",
]
reload_cmd = "/opt/sentinel/confd/scripts/remove-if-empty /opt/sentinel/etc/org.opennms.features.flows.persistence.elastic.cfg"
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[template]
src = "org.opennms.netmgt.distributed.datasource.cfg.tmpl"
dest = "/opt/sentinel/etc/org.opennms.netmgt.distributed.datasource.cfg"
keys = [
"/datasource/url",
"/datasource/username",
"/datasource/password",
"/datasource/database-name",
]
reload_cmd = "/opt/sentinel/confd/scripts/remove-if-empty /opt/sentinel/etc/org.opennms.netmgt.distributed.datasource.cfg"
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[template]
src = "org.opennms.sentinel.controller.cfg.tmpl"
dest = "/opt/sentinel/etc/org.opennms.sentinel.controller.cfg"
keys = [
"/location",
"/id",
"/broker-url"
]
reload_cmd = "/opt/sentinel/confd/scripts/remove-if-empty /opt/sentinel/etc/org.opennms.sentinel.controller.cfg"
4 changes: 4 additions & 0 deletions opennms-container/sentinel/container-fs/confd/confd.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
confdir = "/opt/sentinel/confd"
backend = "file"
file = [ "/opt/sentinel/sentinel-config.yaml" ]
log-level = "debug"
Empty file.
19 changes: 19 additions & 0 deletions opennms-container/sentinel/container-fs/confd/scripts/confd_lib.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
SENTINEL_ETC_DIR="/opt/sentinel/etc"
STATIC_DIR="/opt/sentinel/confd/static"

# Checks if the given file is empty and if so deletes it.
removeIfEmpty() {
local file="$1"

if [ -z "$file" ]; then
echo "No rendered template was specified"
exit 1
fi

if [ -s "$file" ]; then
return 1
fi

rm "$file"
return 0
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
. "/opt/sentinel/confd/scripts/confd_lib.sh"

if removeIfEmpty "$1"; then
echo "Removed file '$1' since it was empty after rendering"
fi
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{{- /* Upon updating this file please ensure that the schema sentinel-config-schema.yml accurately reflects the keys that
are used by this template. */ -}}
{{if ls "/telemetry/flows/adapters"}}
<?xml version="1.0" encoding="UTF-8"?>
<features name="flows-features" xmlns="http://karaf.apache.org/xmlns/features/v1.3.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://karaf.apache.org/xmlns/features/v1.3.0 http://karaf.apache.org/xmlns/features/v1.3.0">

<feature name="confd-flows-auto" version="1.0" install="auto">

{{$flowsPath := "/telemetry/flows" -}}
{{range lsdir (print $flowsPath "/adapters") -}}
{{$adaptersPath := (print $flowsPath "/adapters/" .) -}}
{{$adapterName := base $adaptersPath -}}
<config name="org.opennms.features.telemetry.adapters-{{$adapterName}}">
name={{$adapterName}}
class-name={{getv (print $adaptersPath "/class-name")}}
{{range gets (print $adaptersPath "/parameters/*") -}}
parameters.{{base .Key}}={{.Value}}
{{end -}}
</config>
{{end -}}

</feature>
<feature>sentinel-flows</feature>
<feature>sentinel-jsonstore-postgres</feature>
<feature>sentinel-blobstore-noop</feature>

</features>
{{end -}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{{- /* Upon updating this file please ensure that the schema sentinel-config-schema.yml accurately reflects the keys that
are used by this template. */ -}}
{{if exists "/org.opennms.instance.id" -}}
org.opennms.instance.id = {{getv "/org.opennms.instance.id"}}
{{end -}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{{- /* Upon updating this file please ensure that the schema sentinel-config-schema.yml accurately reflects the keys that
are used by this template. */ -}}
{{if (exists "/ipc/kafka/bootstrap.servers") -}}
!sentinel-jms
sentinel-kafka
{{else}}
sentinel-jms
!sentinel-kafka
{{end -}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{{- /* Upon updating this file please ensure that the schema sentinel-config-schema.yml accurately reflects the keys that
are used by this template. */ -}}
{{range gets "/ipc/kafka/*" -}}
{{base .Key}} = {{.Value}}
{{end -}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{{- /* Upon updating this file please ensure that the schema sentinel-config-schema.yml accurately reflects the keys that
are used by this template. */ -}}
{{range gets "/ipc/kafka/*" -}}
{{base .Key}} = {{.Value}}
{{end -}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{{- /* Upon updating this file please ensure that the schema sentinel-config-schema.yml accurately reflects the keys that
are used by this template. */ -}}
{{if exists "/elasticsearch/url" -}}
elasticUrl = {{getv "/elasticsearch/url"}}
{{end -}}
{{if exists "/elasticsearch/index-strategy" -}}
elasticIndexStrategy = {{getv "/elasticsearch/index-strategy"}}
{{end -}}
{{if exists "/elasticsearch/replicas" -}}
settings.index.number_of_replicas = {{getv "/elasticsearch/replicas"}}
{{end -}}
{{if exists "/elasticsearch/conn-timeout" -}}
connTimeout = {{getv "/elasticsearch/conn-timeout"}}
{{end -}}
{{if exists "/elasticsearch/read-timeout" -}}
readTimeout = {{getv "/elasticsearch/read-timeout"}}
{{end -}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{{- /* Upon updating this file please ensure that the schema sentinel-config-schema.yml accurately reflects the keys that
are used by this template. */ -}}
{{if exists "/datasource/url" -}}
datasource.url = {{getv "/datasource/url"}}
{{end -}}
{{if exists "/datasource/username" -}}
datasource.username = {{getv "/datasource/username"}}
{{end -}}
{{if exists "/datasource/password" -}}
datasource.password = {{getv "/datasource/password"}}
{{end -}}
{{if exists "/datasource/database-name" -}}
datasource.databaseName = {{getv "/datasource/database-name"}}
{{end -}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{{- /* Upon updating this file please ensure that the schema sentinel-config-schema.yml accurately reflects the keys that
are used by this template. */ -}}
{{if exists "/location" -}}
location = {{getv "/location"}}
{{end -}}
{{if exists "/id" -}}
id = {{getv "/id"}}
{{end -}}
{{if exists "/broker-url" -}}
broker-url = {{getv "/broker-url"}}
{{end -}}
27 changes: 27 additions & 0 deletions opennms-container/sentinel/container-fs/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -16,6 +16,10 @@ umask 002
export SENTINEL_HOME="/opt/sentinel"
SENTINEL_OVERLAY_ETC="/opt/sentinel-etc-overlay"
SENTINEL_OVERLAY="/opt/sentinel-overlay"
CONFD_KEY_STORE="${SENTINEL_HOME}/sentinel-config.yaml"
CONFD_CONFIG_DIR="${SENTINEL_HOME}/confd"
CONFD_BIN="/usr/bin/confd"
CONFD_CONFIG_FILE="${CONFD_CONFIG_DIR}/confd.toml"

export KARAF_OPTS="-Djava.locale.providers=CLDR,COMPAT -Djdk.util.zip.disableZip64ExtraFieldValidation=true"

@@ -122,6 +126,15 @@ applyOverlayConfig() {
fi
}

applyConfd() {
if [ -f "${CONFD_KEY_STORE}" ]; then
echo "Found a configuration key store, applying configuration via confd."
runConfd
else
echo "No configuration key store present, skipping confd configuration."
fi
}

applyKarafDebugLogging() {
if [ -n "$KARAF_DEBUG_LOGGING" ]; then
echo "Updating Karaf debug logging"
@@ -142,6 +155,17 @@ start() {
exec ./karaf server ${SENTINEL_DEBUG}
}

runConfd() {
# Create any directories that confd might write to
while IFS= read -r dir; do
local dirToCreate="$SENTINEL_HOME"/"$dir"
echo "Creating $dirToCreate so confd can write to it"
mkdir -p "$dirToCreate"
done < "$CONFD_CONFIG_DIR"/directories

"$CONFD_BIN" -onetime -config-file "$CONFD_CONFIG_FILE"
}

# Evaluate arguments for build script.
if [[ "${#}" == 0 ]]; then
usage
@@ -154,6 +178,7 @@ while getopts csdfh flag; do
c)
useEnvCredentials
initConfig
applyConfd
applyOverlayConfig
applyKarafDebugLogging
start
@@ -164,12 +189,14 @@ while getopts csdfh flag; do
d)
SENTINEL_DEBUG="debug"
initConfig
applyConfd
applyOverlayConfig
applyKarafDebugLogging
start
;;
f)
initConfig
applyConfd
applyOverlayConfig
applyKarafDebugLogging
start
Loading
Loading