Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MQE: Add support to disable functions+aggregations by name #10384

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

jhesketh
Copy link
Contributor

@jhesketh jhesketh commented Jan 9, 2025

Related to #10067

@jhesketh jhesketh requested a review from a team as a code owner January 9, 2025 05:46
@jhesketh jhesketh mentioned this pull request Jan 9, 2025
@jhesketh jhesketh force-pushed the jhesketh/mqe-function-flags branch from 8646304 to 28b5513 Compare January 9, 2025 06:19
@jhesketh jhesketh requested a review from tacole02 as a code owner January 9, 2025 06:19
pkg/streamingpromql/config.go Outdated Show resolved Hide resolved
pkg/streamingpromql/config.go Outdated Show resolved Hide resolved
pkg/streamingpromql/engine_test.go Outdated Show resolved Hide resolved
pkg/streamingpromql/operators/aggregations/common.go Outdated Show resolved Hide resolved
pkg/streamingpromql/query.go Outdated Show resolved Hide resolved
Comment on lines 319 to 320
if e.Func.Name == fName {
return nil, compat.NewNotSupportedError(fmt.Sprintf("'%s' function is disabled", e.Func.Name))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In all other cases, we don't differentiate between a feature being unsupported entirely and a feature being disabled by a feature flag:

Suggested change
if e.Func.Name == fName {
return nil, compat.NewNotSupportedError(fmt.Sprintf("'%s' function is disabled", e.Func.Name))
if e.Func.Name == fName {
return nil, compat.NewNotSupportedError(fmt.Sprintf("'%s' function", e.Func.Name))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this is a little more explicitly disabled so the log line is helpful

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow sorry - isn't it just as explicit as any other feature disabled with a feature flag?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The others are "enable" which default to true. This explicitly disables something, which is useful to know in the logs imo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep in mind that this value is used not just in a log message but also in the reason label for the cortex_mimir_query_engine_unsupported_queries_total metric.

pkg/streamingpromql/query.go Show resolved Hide resolved
Copy link
Contributor

@tacole02 tacole02 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I removed a few instances of "will", as we try to avoid future tense in docs.

pkg/streamingpromql/engine.go Outdated Show resolved Hide resolved
pkg/streamingpromql/engine.go Outdated Show resolved Hide resolved
pkg/streamingpromql/config.go Outdated Show resolved Hide resolved
pkg/streamingpromql/operators/aggregations/common.go Outdated Show resolved Hide resolved
@@ -35,3 +37,27 @@ var AggregationGroupFactories = map[parser.ItemType]AggregationGroupFactory{
//
// Invalid combinations include exponential and custom buckets, and histograms with incompatible custom buckets.
var invalidCombinationOfHistograms = &histogram.FloatHistogram{}

func GetAggregationItemType(aggregation string) (parser.ItemType, bool) {
// The parser key is not exported, but the item types are. It's safe to assume the keys will continue to map their names onto their values.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I follow this comment - are you trying to say that it's safe to assume that (for example) the string sum will continue to represent parser.SUM?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about rephrasing this to something like this then?

Suggested change
// The parser key is not exported, but the item types are. It's safe to assume the keys will continue to map their names onto their values.
// The aggregation names are not exported, but their item types are. It's safe to assume the names will not change.

@@ -44,12 +47,26 @@ func NewEngine(opts EngineOpts, limitsProvider QueryLimitsProvider, metrics *sta
return nil, errors.New("enabling delayed name removal not supported by Mimir query engine")
}

// Sort DisabledFunctions to optimise lookups
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make this comment (and the one below) even clearer: we must sort these two slices because we use a binary search over them later on.

requireQueryIsUnsupported(t, features, "SUM(metric{})", "'sum' aggregation disabled")
})

t.Run("wrong aggregation name disabled", func(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit]

Suggested change
t.Run("wrong aggregation name disabled", func(t *testing.T) {
t.Run("unknown aggregation name disabled", func(t *testing.T) {

@@ -35,3 +37,27 @@ var AggregationGroupFactories = map[parser.ItemType]AggregationGroupFactory{
//
// Invalid combinations include exponential and custom buckets, and histograms with incompatible custom buckets.
var invalidCombinationOfHistograms = &histogram.FloatHistogram{}

func GetAggregationItemType(aggregation string) (parser.ItemType, bool) {
// The parser key is not exported, but the item types are. It's safe to assume the keys will continue to map their names onto their values.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about rephrasing this to something like this then?

Suggested change
// The parser key is not exported, but the item types are. It's safe to assume the keys will continue to map their names onto their values.
// The aggregation names are not exported, but their item types are. It's safe to assume the names will not change.

func GetAggregationItemType(aggregation string) (parser.ItemType, bool) {
// The parser key is not exported, but the item types are. It's safe to assume the keys will continue to map their names onto their values.
// (ie, "avg" will be parser.AVG).
var key = map[string]parser.ItemType{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need to create this map each time GetAggregationItemType is called - what if we move this outside this method?

Comment on lines +316 to +317
// e.Func.Name is already validated by the parser. Meaning we don't need to check if the function name
// is real before checking if it is disabled.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit]

Suggested change
// e.Func.Name is already validated by the parser. Meaning we don't need to check if the function name
// is real before checking if it is disabled.
// e.Func.Name is already validated and canonicalised by the parser. Meaning we don't need to check if the function name
// refers to a function that exists, nor normalise the casing etc. before checking if it is disabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants