feat: Dynatrace scaler #5685

cyrilico · 2024-04-11T14:54:10Z

Add new scaler for interacting with Dynatrace and its Get Metric Data Points API

Checklist

When introducing a new scaler, I agree with the scaling governance policy
I have verified that my change is according to the deprecations & breaking changes policy
Tests have been added
Changelog has been updated and is aligned with our changelog requirements
A PR is opened to update the documentation on (repo) (if applicable)
Commits are signed with Developer Certificate of Origin (DCO - learn more)

Closes #2411

Relates to kedacore/keda-docs#1360

Signed-off-by: cyrilico <[email protected]>

Signed-off-by: damas <[email protected]>

Signed-off-by: cyrilico <[email protected]>

zroubalik · 2024-04-24T19:17:26Z

/run-e2e dynatrace
Update: You can check the progress here

JorTurFer · 2024-04-28T10:15:34Z

Please, rebase your branch @cyrilico , the e2e issue was related with some breaking changes in otel-collector which are already fixed in main branch (your tests hasn't been executed yet indeed)

Signed-off-by: cyrilico <[email protected]>

JorTurFer · 2024-04-28T10:39:47Z

/run-e2e dynatrace
Update: You can check the progress here

tests/scalers/dynatrace/dynatrace_test.go

Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: cyrilico <[email protected]>

Signed-off-by: cyrilico <[email protected]>

JorTurFer · 2024-04-28T14:23:59Z

/run-e2e dynatrace
Update: You can check the progress here

Signed-off-by: cyrilico <[email protected]>

cyrilico · 2024-04-28T17:43:07Z

new changes, please rerun

JorTurFer · 2024-04-28T17:56:12Z

/run-e2e dynatrace
Update: You can check the progress here

tests/scalers/dynatrace/dynatrace_test.go

Signed-off-by: Jorge Turrado Ferrero <[email protected]>

JorTurFer · 2024-04-28T21:20:17Z

/run-e2e dynatrace
Update: You can check the progress here

JorTurFer · 2024-04-28T21:21:07Z

@cyrilico I've merged 2 fixes to the e2e to trigger them again (you'll need to pull the changes :) )

Signed-off-by: cyrilico <[email protected]>

JorTurFer · 2024-05-26T16:06:58Z

Hello,
I've been testing this locally and I found some things:

The dynatrace operator fails because the token doesn't have enough permissions for it
The job generators aren't generating the load because the values are overridden

For the dynatrace operator issue, I've created a new secret 'DYNATRACE_OPERATOR_TOKEN', please use it for the operator and the other one for metrics (including for reading them by KEDA)

After making these changes, I see that dynatrace operator has created more pods:

But the metric isn't still available:

I think that the issue can be in something related with the e2e test configuration because I can see that KEDA is querying the value and getting an empty response with status 200 (so from KEDA side seems that works)

cyrilico · 2024-05-27T16:05:06Z

Appreciate the help @JorTurFer 🙏 What metrics are available in that page? The one I used is supposed to be a built-in, i.e., supported OOTB, so I'm curious as to what exactly is available in that dynatrace environment

Signed-off-by: cyrilico <[email protected]>

cyrilico · 2024-06-22T21:04:37Z

Any news @JorTurFer ? 🙏

JorTurFer · 2024-06-24T12:56:55Z

Any news @JorTurFer ? 🙏

I have to apologize for not answering before, I've been involved on a huge working peak :(
I'll try to review this ASAP. I guess that the issue with the metrics was that they needed more time to be consolidated, maybe we can use an approach like the Azure Monitor one, just querying about any available metric as default and updating the ScaledObject to trigger the scaling. e.g: Imagine that you can return an arbitrary number and you use it for scaling, at the end of the day, we don't need to check that the vendor calculates correctly the metric but just the connectivity

cyrilico · 2024-06-24T20:24:47Z

Thanks, I have noticed in the corporate instance I have access to that Dynatrace does take a minute (sometimes literally) to ingest a data point, so I believe that might be an interesting approach. I'll take a look at the Azure Monitor scaler and try to get some inspiration

JorTurFer · 2024-06-24T20:33:33Z

For instance, influx does it too: https://github.com/kedacore/keda/blob/main/tests/scalers/influxdb/influxdb_test.go#L106-L117

It sets the expected value within the query and just updating the ScaledObject you change the received value.

And this is the Azure log analytics one (I said azure monitor worngly): https://github.com/kedacore/keda/blob/main/tests/scalers/azure/azure_log_analytics/azure_log_analytics_test.go#L113-L123

Signed-off-by: cyrilico <[email protected]>

cyrilico · 2024-07-21T19:11:38Z

@JorTurFer couldn't find an exact map to replicate that mechanism in Dynatrace (other than potentially mocking the API calls, which would invalidate the connection testing part), so I've tried a configurable default value filter available in their query language spec; can we have another e2e test run?

JorTurFer · 2024-07-25T09:59:28Z

/run-e2e dynatrace
Update: You can check the progress here

JorTurFer · 2024-07-25T10:00:41Z

Hello,
I'm so sorry due to the long silent from my side and I'd like to apologize, my life's been complicated during last months. During this week and next week I'll be focus on this PR (and others) aiming to include them within next week release.
Thanks for your contribution :)

Signed-off-by: Jorge Turrado <[email protected]>

JorTurFer · 2024-07-25T14:31:39Z

/run-e2e dynatrace
Update: You can check the progress here

JorTurFer · 2024-07-25T14:33:18Z

@cyrilico , I've updated the e2e test to use custom metrics. We have the full control over them and they don't need the agent or any other service, just POST to ingest and KEDA will read them 😄
It's the same approach that we use for Azure Application Insights

Signed-off-by: Jorge Turrado <[email protected]>

JorTurFer · 2024-07-25T15:01:53Z

/run-e2e dynatrace
Update: You can check the progress here

JorTurFer

LGTM! Thanks for the contribution! ❤️
PTAL @zroubalik

zroubalik

This is really great, there is one minor thing, do you think you can refactor the scaler metadata parsing (parseDynatraceMetadata())? We introduced a new simplified way of parsing, it would be great to include this change in this scaler. You can see details at #5797

Signed-off-by: cyrilico <[email protected]>

cyrilico · 2024-07-30T10:30:05Z

done @zroubalik 👌

JorTurFer · 2024-07-30T15:39:25Z

/run-e2e dynatrace
Update: You can check the progress here

zroubalik

thanks @cyrilico !

satrox28 · 2024-08-10T10:34:18Z

Thank you, makes life easier now. Don't have to deploy Prometheus server additionally on each cluster.

* Add first scaler version Signed-off-by: cyrilico <[email protected]> * small refactor for response validation Signed-off-by: cyrilico <[email protected]> * Add 'from' property, rename host/token Signed-off-by: cyrilico <[email protected]> * Add parsing tests Signed-off-by: cyrilico <[email protected]> * update changelog Signed-off-by: cyrilico <[email protected]> * Update CHANGELOG.md Signed-off-by: damas <[email protected]> * Update values type to float64 Signed-off-by: damas <[email protected]> * Remove unnecessary conversion Signed-off-by: damas <[email protected]> * e2e tests Signed-off-by: cyrilico <[email protected]> * Apply suggestions from code review Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: cyrilico <[email protected]> * Update dynatrace_test.go Signed-off-by: cyrilico <[email protected]> * Fix bad templating for e2e tests Signed-off-by: cyrilico <[email protected]> * Revert unnecessary (?) template variable change Signed-off-by: cyrilico <[email protected]> * Apply suggestions from code review Signed-off-by: Jorge Turrado Ferrero <[email protected]> * Update tests/scalers/dynatrace/dynatrace_test.go Signed-off-by: Jorge Turrado Ferrero <[email protected]> * Do not allow token to be passed in scaledobject trigger Signed-off-by: cyrilico <[email protected]> * Remove bad secret, tweak dynakube test config Signed-off-by: cyrilico <[email protected]> * Rename property in response parsing Signed-off-by: cyrilico <[email protected]> * Update tests/scalers/dynatrace/dynatrace_test.go Signed-off-by: Jorge Turrado Ferrero <[email protected]> * use new operator secret, update template variable naming Signed-off-by: cyrilico <[email protected]> * forgotten correct variable definition Signed-off-by: cyrilico <[email protected]> * try default value in query for e2e tests Signed-off-by: cyrilico <[email protected]> * fix missing closing parenthesis, bad indenting Signed-off-by: cyrilico <[email protected]> * Update e2e test to use custom metrics Signed-off-by: Jorge Turrado <[email protected]> * Close the body to fix static checks Signed-off-by: Jorge Turrado <[email protected]> * use declarative scaler config Signed-off-by: cyrilico <[email protected]> --------- Signed-off-by: cyrilico <[email protected]> Signed-off-by: damas <[email protected]> Signed-off-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: Jorge Turrado <[email protected]> Co-authored-by: Jorge Turrado Ferrero <[email protected]> Co-authored-by: Jorge Turrado <[email protected]> Signed-off-by: Jorge Turrado <[email protected]>

cyrilico added 9 commits April 11, 2024 15:53

Add first scaler version

7c81b45

Signed-off-by: cyrilico <[email protected]>

small refactor for response validation

93e3270

Signed-off-by: cyrilico <[email protected]>

Add 'from' property, rename host/token

5a77a72

Signed-off-by: cyrilico <[email protected]>

Add parsing tests

0456bc7

Signed-off-by: cyrilico <[email protected]>

update changelog

402e2e8

Signed-off-by: cyrilico <[email protected]>

Merge branch 'main' into dynatrace_scaler

efca4d7

Signed-off-by: damas <[email protected]>

Update CHANGELOG.md

2583560

Signed-off-by: damas <[email protected]>

Update values type to float64

f7661de

Signed-off-by: damas <[email protected]>

Remove unnecessary conversion

3b4ca9d

Signed-off-by: damas <[email protected]>

cyrilico mentioned this pull request Apr 12, 2024

Add Dynatrace scaler docs kedacore/keda-docs#1360

Merged

1 task

e2e tests

4e3d039

Signed-off-by: cyrilico <[email protected]>

cyrilico marked this pull request as ready for review April 24, 2024 14:40

cyrilico requested a review from a team as a code owner April 24, 2024 14:40

Merge branch 'main' into dynatrace_scaler

7cf6d2b

Signed-off-by: cyrilico <[email protected]>

JorTurFer reviewed Apr 28, 2024

View reviewed changes

tests/scalers/dynatrace/dynatrace_test.go Outdated Show resolved Hide resolved

tests/scalers/dynatrace/dynatrace_test.go Outdated Show resolved Hide resolved

tests/scalers/dynatrace/dynatrace_test.go Outdated Show resolved Hide resolved

cyrilico and others added 2 commits April 28, 2024 14:48

Apply suggestions from code review

911b0df

Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: cyrilico <[email protected]>

Update dynatrace_test.go

698d6be

Signed-off-by: cyrilico <[email protected]>

cyrilico requested a review from JorTurFer April 28, 2024 14:03

cyrilico added 2 commits April 28, 2024 18:13

Fix bad templating for e2e tests

622266e

Signed-off-by: cyrilico <[email protected]>

Revert unnecessary (?) template variable change

e0d42eb

Signed-off-by: cyrilico <[email protected]>

JorTurFer reviewed Apr 28, 2024

View reviewed changes

tests/scalers/dynatrace/dynatrace_test.go Outdated Show resolved Hide resolved

tests/scalers/dynatrace/dynatrace_test.go Outdated Show resolved Hide resolved

Apply suggestions from code review

607805e

Signed-off-by: Jorge Turrado Ferrero <[email protected]>

Merge branch 'main' into dynatrace_scaler

de19f06

Signed-off-by: cyrilico <[email protected]>

cyrilico added 2 commits May 27, 2024 17:05

use new operator secret, update template variable naming

fb5ce7b

Signed-off-by: cyrilico <[email protected]>

forgotten correct variable definition

6e03f9b

Signed-off-by: cyrilico <[email protected]>

cyrilico added 3 commits July 20, 2024 19:57

Merge branch 'main' into dynatrace_scaler

3505f51

Signed-off-by: cyrilico <[email protected]>

try default value in query for e2e tests

b40f11f

Signed-off-by: cyrilico <[email protected]>

fix missing closing parenthesis, bad indenting

fd857af

Signed-off-by: cyrilico <[email protected]>

Update e2e test to use custom metrics

d6971dc

Signed-off-by: Jorge Turrado <[email protected]>

Close the body to fix static checks

54428f7

Signed-off-by: Jorge Turrado <[email protected]>

JorTurFer approved these changes Jul 25, 2024

View reviewed changes

zroubalik reviewed Jul 30, 2024

View reviewed changes

use declarative scaler config

e92d83f

Signed-off-by: cyrilico <[email protected]>

cyrilico requested a review from zroubalik July 30, 2024 10:29

JorTurFer merged commit 12a529d into kedacore:main Jul 30, 2024
18 checks passed

zroubalik reviewed Jul 31, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Dynatrace scaler #5685

feat: Dynatrace scaler #5685

cyrilico commented Apr 11, 2024 •

edited

Loading

zroubalik commented Apr 24, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Apr 28, 2024

JorTurFer commented Apr 28, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Apr 28, 2024 •

edited by github-actions bot

Loading

cyrilico commented Apr 28, 2024

JorTurFer commented Apr 28, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Apr 28, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Apr 28, 2024

JorTurFer commented May 26, 2024

cyrilico commented May 27, 2024

cyrilico commented Jun 22, 2024

JorTurFer commented Jun 24, 2024

cyrilico commented Jun 24, 2024

JorTurFer commented Jun 24, 2024

cyrilico commented Jul 21, 2024

JorTurFer commented Jul 25, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jul 25, 2024

JorTurFer commented Jul 25, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jul 25, 2024 •

edited

Loading

JorTurFer commented Jul 25, 2024 •

edited by github-actions bot

Loading

JorTurFer left a comment

zroubalik left a comment •

edited

Loading

cyrilico commented Jul 30, 2024

JorTurFer commented Jul 30, 2024 •

edited by github-actions bot

Loading

zroubalik left a comment

satrox28 commented Aug 10, 2024 •

edited

Loading

feat: Dynatrace scaler #5685

feat: Dynatrace scaler #5685

Conversation

cyrilico commented Apr 11, 2024 • edited Loading

Checklist

zroubalik commented Apr 24, 2024 • edited by github-actions bot Loading

JorTurFer commented Apr 28, 2024

JorTurFer commented Apr 28, 2024 • edited by github-actions bot Loading

JorTurFer commented Apr 28, 2024 • edited by github-actions bot Loading

cyrilico commented Apr 28, 2024

JorTurFer commented Apr 28, 2024 • edited by github-actions bot Loading

JorTurFer commented Apr 28, 2024 • edited by github-actions bot Loading

JorTurFer commented Apr 28, 2024

JorTurFer commented May 26, 2024

cyrilico commented May 27, 2024

cyrilico commented Jun 22, 2024

JorTurFer commented Jun 24, 2024

cyrilico commented Jun 24, 2024

JorTurFer commented Jun 24, 2024

cyrilico commented Jul 21, 2024

JorTurFer commented Jul 25, 2024 • edited by github-actions bot Loading

JorTurFer commented Jul 25, 2024

JorTurFer commented Jul 25, 2024 • edited by github-actions bot Loading

JorTurFer commented Jul 25, 2024 • edited Loading

JorTurFer commented Jul 25, 2024 • edited by github-actions bot Loading

JorTurFer left a comment

Choose a reason for hiding this comment

zroubalik left a comment • edited Loading

Choose a reason for hiding this comment

cyrilico commented Jul 30, 2024

JorTurFer commented Jul 30, 2024 • edited by github-actions bot Loading

zroubalik left a comment

Choose a reason for hiding this comment

satrox28 commented Aug 10, 2024 • edited Loading

cyrilico commented Apr 11, 2024 •

edited

Loading

zroubalik commented Apr 24, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Apr 28, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Apr 28, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Apr 28, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Apr 28, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jul 25, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jul 25, 2024 •

edited by github-actions bot

Loading

JorTurFer commented Jul 25, 2024 •

edited

Loading

JorTurFer commented Jul 25, 2024 •

edited by github-actions bot

Loading

zroubalik left a comment •

edited

Loading

JorTurFer commented Jul 30, 2024 •

edited by github-actions bot

Loading

satrox28 commented Aug 10, 2024 •

edited

Loading