Grafana integration with existing prometheus/grafana docker "stack"

[ninjamonkey198206]

The current method looks like it involves a baremetal t-rmm install, using the provided script to install grafana alongside t-rmm.

So far I haven't been able to work through the script to see if I could pull the relevant info to try to create a functional config for t-rmm and grafana running via Docker.

@dinger1986 , it looks like this was your baby. Do you think it's possible to work out a way to do it?

[ninjamonkey198206]

I may have found most of what I needed in the script. Will update with progress.

[ninjamonkey198206]

Works perfectly on my home setup (aside from the map, that's still in progress) but doesn't automatically select the appropriate agent when used from my Google VM. It's likely due to my use of Drop All by default IPTables rules and HAProxy, requiring everything to be bound to localhost rather than publicly exposed.

I'll add an update for that edge case situation once I work it out. It's likely just a tweak to the IPTables rules. Docker networking gets twitchy with that setup, requiring some extra rules to make it play nice.

Edit: Solved. I didn't notice that when I copied, pasted, and edited the URL that it had done something funky and messed it up. Correcting the URL syntax solved the issue.

So, the only quirk left to solve aside from the map is the requirement for it to be the default datasource, as in my case, that gets overridden and the Prometheus datasource takes over every time the stack is cycled. It's a minor annoyance though, as I just have to go into the Tactical datasource and set it back to default.

[ninjamonkey198206]

@dinger1986 Adding a datasource variable to the Tactical TRMM dashboard and editing it to select the Tactical postgres datasource seems to eliminate the need for the Tactical datasource to be the default. Unfortunately, I'm having to go through and change each panel to get it to use the variable. I'll fork your repo and upload the modified versions once I'm done.

[ninjamonkey198206]

https://github.com/ninjamonkey198206/TRMM-Grafana
The link to the forked repo. Cleaned up some minor typos and changed some of the titles too. Let me know what you think.

[ninjamonkey198206]

@dinger1986 I did a thing! Copied your original json files to new files in the repo I forked from yours. Played a bit, changed some titles and descriptions since there are now Linux agents, added the data source variable so the T-RMM data source doesn't need to be the default, moved stuff around so the agent has its own page as does the client/site, and added dashboard links to them all so you can flip back and forth direct from the dashboards!

Woohoo!

I may be a little excited because this is the first time I've edited dashboards, and they actually worked when I was done.

I'm hoping I can add more functions to the agent dashboard, like network traffic, I/O, etc. If you wouldn't mind, I could use some help on the client dashboard, as I duplicated the "client" panel and I'm hoping to have one be for the selected clients, one for sites for the selected clients, and one for agents for the selected clients/sites.

I don't know if it's possible to do securely as it would be cross site, but with the links between the dashboards on the dashboards themselves, maybe the agent panel could be an option to embedded into the bottom section of T-RMM, along with the Summary, Checks, Tasks, and such.

[dinger1986]

Sounds really cool, feel free to do a PR, tbh I just facilitated an easy install method for others, Hugo and Sebcashmag on discord are much more knowledgeable about this than I am.