Prometheus Adapter Error High Availability

[ips219]

bitnami/influxdb-4.8.9

I've deployed Prometheus and InfluxDB together in a EKS cluster and it seems that there is a problem when remote_write is enabled in Prometheus in order to publish metrics to InfluxDB.

In this scenario, the logs of Prometheus display this message: server returned HTTP status 404 Not Found: {"error":"invalid write endpoint"}

As I understand, the root cause may be this (influxdata/influxdb-relay#67), it seems influxdb-relay is not maintained anymore by influxdata (I guess because they offer HA support in InfluxDB Enterprise)

Also I´ve noticed that people from veepee-oss (https://github.com/veepee-oss/influxdb-relay) has forked influxdb-relay project in order to mantain it, and it seems they fixed this issue among others.

I was wondering if you have any plan to change the base image used for building bitnami influxdb image and using the new one mantained by veepee-oss?

[miguelaeh]

Hi @ips219 ,
Thank you for the information. I am not able to find an unmaintained warning or something, could you point me to it please?

[ips219]

Hi Miguel...

To be honest I'm not an active follower of InfluxDB project, just noticed on the issue comments (influxdata/influxdb-relay#66) and the pending merge requests (https://github.com/influxdata/influxdb-relay/pulls).. Any of them seems to have any comment from the authors since 2016. As I said, I guess it is normal they don´t want to maintain the HA component actively considering they have an enterprise licensed version...

I am evaluating the HA solution Prometheus Chart + InfluxDB Chart among others (e.g Cortex) and I arrived to this issue...

Just for the records and considering the High Availability arquitecture deployed by the chart (influxdb-relay pods just broadcast all write requests received to all running InfluxDB PODS), for the time being I just bypassed relay component and updated all cluster POD instances manually from Prometheus directly using the provided headless service endpoints in the configuration values of Prometheus chart.

remote_write: - url: http://demo-influxdb-0.demo-influxdb-headless.influxdb.svc.cluster.local:8086/api/v1/prom/write?db=prometheus - url: http://demo-influxdb-1.demo-influxdb-headless.influxdb.svc.cluster.local:8086/api/v1/prom/write?db=prometheus
Not perfect as it is an overhead for Prometheus... but It works as an easy workaround.

[miguelaeh]

Hi @ips219 ,
I am glad you found a workaround, thank you for sharing it.
About the unmaintained repo, I am not sure, it seems the latest commit was 3 months ago.

[stale]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[stale]

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.