Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sotw: rlp workflow #893

Draft
wants to merge 39 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
c62946d
sotw: effective ratelimitpolicy
guicassolato Sep 27, 2024
0241d73
removed unused function to apply rlp overrides
guicassolato Oct 8, 2024
556cbf1
istio extension (WasmPlugin) reconciler
guicassolato Oct 8, 2024
e95e8eb
fixup: unique limit definitions per scope
guicassolato Oct 8, 2024
b49a6e2
fixup: log error only when indeed there's an error to be logged
guicassolato Oct 8, 2024
d625294
do not fail when missing kuadrant object
guicassolato Oct 8, 2024
ed863c6
fixup: equality between wasmplugins and avoid rebuilding wasm config …
guicassolato Oct 8, 2024
1eda5ed
fixup: error comparison
guicassolato Oct 8, 2024
923a0db
cleanup istio extension objects when it cannot calculate effective po…
guicassolato Oct 8, 2024
14c5398
refactor: removal of SortablePolicy for sorting policies objects by c…
guicassolato Oct 9, 2024
4b25025
remove no longer relevant integration test case
guicassolato Oct 9, 2024
fccb02d
fixup: avoid updating invalid rate limit policies to 'accepted' on ev…
guicassolato Oct 9, 2024
5da43e5
fixup: continue istio extension workflow when it fails for a given ga…
guicassolato Oct 10, 2024
5aaa5d7
Remove unnused event recorder from base reconciler
guicassolato Oct 10, 2024
e6e7128
Istio rate limit cluster reconciler
guicassolato Oct 10, 2024
ae357e2
enable Istio-related rate limit tasks only when Istio is installed
guicassolato Oct 10, 2024
19dc919
ensure at least one hostname per wasm config policy
guicassolato Oct 10, 2024
774707b
bump istio to 1.22
guicassolato Oct 11, 2024
6fd9e62
Use targetRefs to attach to gateways in the Istio EnvoyFilter and Was…
guicassolato Oct 11, 2024
d7efc44
refactor: debug log messages for when Limitador, EnvoyFilter and Wasm…
guicassolato Oct 12, 2024
8865452
sort wasm 'policies' within the wasm plugin config by hostname from m…
guicassolato Oct 12, 2024
dde2c89
go fmt: refactor: debug log messages for when Limitador, EnvoyFilter …
guicassolato Oct 12, 2024
c84b653
sort wasm 'policies' within the wasm plugin config by hostname and ht…
guicassolato Oct 12, 2024
f1a9669
code style: remove unused parameter ctx
guicassolato Oct 12, 2024
d052f9e
fix: unit test: sort wasm 'policies' within the wasm plugin config by…
guicassolato Oct 12, 2024
cf44f68
Refactor WasmPlugin reconciliation to reduce repetition with EnvoyExt…
guicassolato Oct 14, 2024
3735abc
envoy gateway rate limit cluster reconciler
guicassolato Oct 14, 2024
664634a
fix: wrong default GroupKind assumed on Istio policies TargetRefs
guicassolato Oct 15, 2024
a23579d
minor fixes to code comments and log messages
guicassolato Oct 15, 2024
c21b494
envoy gateway extension reconciler
guicassolato Oct 15, 2024
a3e4b5d
code style: ratelimit.RateLimitIndex -> Index
guicassolato Oct 15, 2024
6ab3ec8
code style: wasm.WasmExtensionName -> ExtensionName
guicassolato Oct 15, 2024
1e9829e
code style: wasm.WasmRuleBuilderFunc -> RuleBuilderFunc
guicassolato Oct 15, 2024
9d7bcd1
code style: fix grouping of go imports
guicassolato Oct 15, 2024
e3d5387
new structure of the wasm config based on https://github.com/Kuadrant…
guicassolato Oct 15, 2024
f7b6a29
fix: envoy_gateway_extension_reconciler.go file name
guicassolato Oct 16, 2024
be54de3
fix: log messages of envoy patch policy reconciliation
guicassolato Oct 16, 2024
5c6ccbc
rlp enforced condition and consistency in the message when target not…
guicassolato Oct 16, 2024
5f1109a
tests: gateway extension reconciler tests resource comparison
guicassolato Oct 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
202 changes: 202 additions & 0 deletions api/v1/merge_strategies.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
/*
Copyright 2024.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1

import (
"crypto/sha256"
"encoding/hex"
"sort"
"strings"

"github.com/samber/lo"
"k8s.io/apimachinery/pkg/runtime"

"github.com/kuadrant/policy-machinery/controller"
"github.com/kuadrant/policy-machinery/machinery"
)

const (
AtomicMergeStrategy = "atomic"
PolicyRuleMergeStrategy = "merge"
)

type MergeableRule struct {
Spec any
Source string
}

// +kubebuilder:object:generate=false
type MergeablePolicy interface {
machinery.Policy

Rules() map[string]MergeableRule
SetRules(map[string]MergeableRule)
Empty() bool

DeepCopyObject() runtime.Object
}

// AtomicDefaultsMergeStrategy implements a merge strategy that returns the target Policy if it exists,
// otherwise it returns the source Policy.
func AtomicDefaultsMergeStrategy(source, target machinery.Policy) machinery.Policy {
if source == nil {
return target

Check warning on line 57 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L55-L57

Added lines #L55 - L57 were not covered by tests
}
if target == nil {
return source

Check warning on line 60 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L59-L60

Added lines #L59 - L60 were not covered by tests
}

mergeableTargetPolicy := target.(MergeablePolicy)

Check warning on line 63 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L63

Added line #L63 was not covered by tests

if !mergeableTargetPolicy.Empty() {
return mergeableTargetPolicy.DeepCopyObject().(machinery.Policy)

Check warning on line 66 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L65-L66

Added lines #L65 - L66 were not covered by tests
}

return source.(MergeablePolicy).DeepCopyObject().(machinery.Policy)

Check warning on line 69 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L69

Added line #L69 was not covered by tests
}

var _ machinery.MergeStrategy = AtomicDefaultsMergeStrategy

// AtomicOverridesMergeStrategy implements a merge strategy that overrides a target Policy with
// a source one.
func AtomicOverridesMergeStrategy(source, _ machinery.Policy) machinery.Policy {
if source == nil {
return nil

Check warning on line 78 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L76-L78

Added lines #L76 - L78 were not covered by tests
}
return source.(MergeablePolicy).DeepCopyObject().(machinery.Policy)

Check warning on line 80 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L80

Added line #L80 was not covered by tests
}

var _ machinery.MergeStrategy = AtomicOverridesMergeStrategy

// PolicyRuleDefaultsMergeStrategy implements a merge strategy that merges a source Policy into a target one
// by keeping the policy rules from the target and adding the ones from the source that do not exist in the target.
func PolicyRuleDefaultsMergeStrategy(source, target machinery.Policy) machinery.Policy {
if source == nil {
return target

Check warning on line 89 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L87-L89

Added lines #L87 - L89 were not covered by tests
}
if target == nil {
return source

Check warning on line 92 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L91-L92

Added lines #L91 - L92 were not covered by tests
}

sourceMergeablePolicy := source.(MergeablePolicy)
targetMergeablePolicy := target.(MergeablePolicy)

Check warning on line 96 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L95-L96

Added lines #L95 - L96 were not covered by tests

// copy rules from the target
rules := targetMergeablePolicy.Rules()

Check warning on line 99 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L99

Added line #L99 was not covered by tests

// add extra rules from the source
for ruleID, rule := range sourceMergeablePolicy.Rules() {
if _, ok := targetMergeablePolicy.Rules()[ruleID]; !ok {
rules[ruleID] = MergeableRule{
Spec: rule.Spec,
Source: source.GetLocator(),

Check warning on line 106 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L102-L106

Added lines #L102 - L106 were not covered by tests
}
}
}

mergedPolicy := targetMergeablePolicy.DeepCopyObject().(MergeablePolicy)
mergedPolicy.SetRules(rules)
return mergedPolicy

Check warning on line 113 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L111-L113

Added lines #L111 - L113 were not covered by tests
}

var _ machinery.MergeStrategy = PolicyRuleDefaultsMergeStrategy

// PolicyRuleOverridesMergeStrategy implements a merge strategy that merges a source Policy into a target one
// by using the policy rules from the source and keeping from the target only the policy rules that do not exist in
// the source.
func PolicyRuleOverridesMergeStrategy(source, target machinery.Policy) machinery.Policy {
sourceMergeablePolicy := source.(MergeablePolicy)
targetMergeablePolicy := target.(MergeablePolicy)

Check warning on line 123 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L121-L123

Added lines #L121 - L123 were not covered by tests

// copy rules from the source
rules := sourceMergeablePolicy.Rules()

Check warning on line 126 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L126

Added line #L126 was not covered by tests

// add extra rules from the target
for ruleID, rule := range targetMergeablePolicy.Rules() {
if _, ok := sourceMergeablePolicy.Rules()[ruleID]; !ok {
rules[ruleID] = rule

Check warning on line 131 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L129-L131

Added lines #L129 - L131 were not covered by tests
}
}

mergedPolicy := targetMergeablePolicy.DeepCopyObject().(MergeablePolicy)
mergedPolicy.SetRules(rules)
return mergedPolicy

Check warning on line 137 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L135-L137

Added lines #L135 - L137 were not covered by tests
}

var _ machinery.MergeStrategy = PolicyRuleOverridesMergeStrategy

func DefaultsMergeStrategy(strategy string) machinery.MergeStrategy {
switch strategy {
case AtomicMergeStrategy:
return AtomicDefaultsMergeStrategy
case PolicyRuleMergeStrategy:
return PolicyRuleDefaultsMergeStrategy
default:
return AtomicDefaultsMergeStrategy

Check warning on line 149 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L142-L149

Added lines #L142 - L149 were not covered by tests
}
}

func OverridesMergeStrategy(strategy string) machinery.MergeStrategy {
switch strategy {
case AtomicMergeStrategy:
return AtomicOverridesMergeStrategy
case PolicyRuleMergeStrategy:
return PolicyRuleOverridesMergeStrategy
default:
return AtomicOverridesMergeStrategy

Check warning on line 160 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L153-L160

Added lines #L153 - L160 were not covered by tests
}
}

// EffectivePolicyForPath returns the effective policy for a given path, merging all policies in the path.
// The policies in the path are sorted from the least specific to the most specific.
// Only policies whose predicate returns true are considered.
func EffectivePolicyForPath[T machinery.Policy](path []machinery.Targetable, predicate func(machinery.Policy) bool) *T {
policies := PoliciesInPath(path, predicate)
if len(policies) == 0 {
return nil

Check warning on line 170 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L167-L170

Added lines #L167 - L170 were not covered by tests
}

// map reduces the policies from most specific to least specific, merging them into one effective policy
effectivePolicy := lo.ReduceRight(policies, func(effectivePolicy machinery.Policy, policy machinery.Policy, _ int) machinery.Policy {
return effectivePolicy.Merge(policy)
}, policies[len(policies)-1])

Check warning on line 176 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L174-L176

Added lines #L174 - L176 were not covered by tests

concreteEffectivePolicy, _ := effectivePolicy.(T)
return &concreteEffectivePolicy

Check warning on line 179 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L178-L179

Added lines #L178 - L179 were not covered by tests
}

// OrderedPoliciesForPath gathers all policies in a path sorted from the least specific to the most specific.
// Only policies whose predicate returns true are considered.
func PoliciesInPath(path []machinery.Targetable, predicate func(machinery.Policy) bool) []machinery.Policy {
return lo.FlatMap(path, func(targetable machinery.Targetable, _ int) []machinery.Policy {
policies := lo.FilterMap(targetable.Policies(), func(policy machinery.Policy, _ int) (controller.Object, bool) {
o, object := policy.(controller.Object)
return o, object && predicate(policy)
})
sort.Sort(controller.ObjectsByCreationTimestamp(policies))
return lo.Map(policies, func(policy controller.Object, _ int) machinery.Policy {
p, _ := policy.(machinery.Policy)
return p
})

Check warning on line 194 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L184-L194

Added lines #L184 - L194 were not covered by tests
})
}

func PathID(path []machinery.Targetable) string {
s := strings.Join(lo.Map(path, machinery.MapTargetableToLocatorFunc), ">")
hash := sha256.Sum256([]byte(s))
return hex.EncodeToString(hash[:8])

Check warning on line 201 in api/v1/merge_strategies.go

View check run for this annotation

Codecov / codecov/patch

api/v1/merge_strategies.go#L198-L201

Added lines #L198 - L201 were not covered by tests
}
32 changes: 21 additions & 11 deletions api/v1beta3/groupversion_info.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
Copyright 2021.
Copyright 2024.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand All @@ -14,23 +14,33 @@
limitations under the License.
*/

// Package v1beta3 contains API Schema definitions for the kuadrant v1beta3 API group
// API schema definitions for the Kuadrant v1beta3 API group
// +kubebuilder:object:generate=true
// +groupName=kuadrant.io
package v1beta3

import (
"k8s.io/apimachinery/pkg/runtime/schema"
"sigs.k8s.io/controller-runtime/pkg/scheme"
ctrl "sigs.k8s.io/controller-runtime/pkg/scheme"
)

var (
// GroupVersion is group version used to register these objects
GroupVersion = schema.GroupVersion{Group: "kuadrant.io", Version: "v1beta3"}
// GroupName specifies the group name used to register the objects.
const GroupName = "kuadrant.io"

// SchemeBuilder is used to add go types to the GroupVersionKind scheme
SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}
// GroupVersion specifies the group and the version used to register the objects.
var GroupVersion = schema.GroupVersion{Group: GroupName, Version: "v1beta3"}

// AddToScheme adds the types in this group-version to the given scheme.
AddToScheme = SchemeBuilder.AddToScheme
)
// SchemeGroupVersion is group version used to register these objects
// Deprecated: use GroupVersion instead.
var SchemeGroupVersion = schema.GroupVersion{Group: GroupName, Version: "v1beta3"}

// SchemeBuilder is used to add go types to the GroupVersionKind scheme
var SchemeBuilder = &ctrl.Builder{GroupVersion: GroupVersion}

// AddToScheme adds the types in this group-version to the given scheme.
var AddToScheme = SchemeBuilder.AddToScheme

// Resource takes an unqualified resource and returns a Group qualified GroupResource
func Resource(resource string) schema.GroupResource {
return SchemeGroupVersion.WithResource(resource).GroupResource()

Check warning on line 45 in api/v1beta3/groupversion_info.go

View check run for this annotation

Codecov / codecov/patch

api/v1beta3/groupversion_info.go#L44-L45

Added lines #L44 - L45 were not covered by tests
}
Loading
Loading