Feature Request: allow K8s secret for Elastalert rules in Helm chart

[cookandy]

Hello,

I think it would be useful to allow reading the Elastalert rules from a secret. This will allow us to keep things like Slack tokens private.

Thoughts?

[jertel]

Seems like a good idea for those that need it. Are you able to add the secret_rules.yaml, and make the volume mount in deployment.yaml so that it mounts the secret file into the container? And document it in the README.md? If so, create the PR and I'll review it/merge it.

[cookandy]

I was originally thinking it would be possible to mount the secret as a volume, and configure Elastalert to use it as one of its rule directories. In order to do this, we would need to mount the volume under /opt/rules and enable scan_subdirectories: true.

However, it appears there's a bug with scan_subdirectories: true, as the current configMap creates some hidden symlinks to the mounted volumes

/opt/rules # ls -lah
total 0
drwxrwxrwx    3 root     root          85 Sep 14 21:40 .
drwxr-xr-x    1 root     root          33 Sep 14 21:40 ..
drwxr-xr-x    2 root     root          32 Sep 14 21:40 ..2020_09_14_21_40_32.208776363
lrwxrwxrwx    1 root     root          31 Sep 14 21:40 ..data -> ..2020_09_14_21_40_32.208776363
lrwxrwxrwx    1 root     root          25 Sep 14 21:40 deadman_slack.yaml -> ..data/deadman_slack.yaml

Which causes Elastalert to fail with the following exception, as it tries to load the same rules from both places

Traceback (most recent call last):                                                                                                                 
   File "/usr/local/lib/python3.6/site-packages/elastalert/loaders.py", line 124, in load                                                           
     raise EAException('Duplicate rule named %s' % (rule['name']))                                                                                  
 elastalert.util.EAException: Duplicate rule named Deadman Switch Slack

Not sure of the best way to proceed.

[jertel]

You could make it an either/or choice: Either the admin can choose to install rules from the secret, or install rules from the config map, but not both. In the deployment.yaml you would do this with code similar to:

{{- if .Values.elasticsearch.secretRulesName }}
        - name: secret-rules
          secret:
            secretName: secret-rules
            items:
            - key: all-secret-rules
              path: all-secret-rules.yaml
{{-end }}

And do the simlar if/end wrapper around the config map volume/mount. There are likely other solutions that might be better, but this would eliminate the need of turning on the scan_subdirectories:true.

[cookandy]

#33 is available for review. Thanks!