sotw: rlp workflow

[guicassolato]

State-of-the-world reconciler – RLP workflow

• spec.targetRef.sectionName
• spec.(defaults|overrides).strategy
• Defaults & Overrides merge strategies (RFC-0009)
• Effective RateLimitPolicy
• Limitador CR (limits)
• Istio cluster (EnvoyFilter)
• istio extension (WasmPlugin)
• envoy cluster (EnvoyPatchPolicy)
• envoy extension (EnvoyExtensionPolicy)
• Accepted status condition
• Enforced status condition
• Split the workflow tasks into separate files

Closes #821

Verification steps

Setup the environment:

make local-setup

(Until Kuadrant/wasm-shim#110 is merged) Patch the deployment to enable the new wasm-shim config structure:

kubectl set env -n kuadrant-system deployment/kuadrant-operator-controller-manager RELATED_IMAGE_WASMSHIM=quay.io/kuadrant/wasm-shim:config-revamp

Enable Envoy Gateway alongside with Istio:

make envoy-gateway-install

kubectl apply -f -<<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: envoygateway
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
EOF

kubectl apply -n gateway-system -f -<<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kuadrant-envoygateway
spec:
  gatewayClassName: envoygateway
  listeners:
  - name: http
    hostname: "*.eg.apps.io"
    protocol: HTTP
    port: 80
    allowedRoutes:
      namespaces:
        from: Same
EOF

Deploy an application:

kubectl apply -f examples/toystore/toystore.yaml

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: toystore
spec:
  parentRefs:
  - name: kuadrant-ingressgateway
    namespace: gateway-system
  - name: kuadrant-envoygateway
    namespace: gateway-system
  rules:
  - backendRefs:
    - name: toystore
      port: 80
    matches:
    - method: GET
  - backendRefs:
    - name: toystore
      port: 80
    matches:
    - method: POST
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: other
spec:
  hostnames:
  - other.example.com
  parentRefs:
  - name: kuadrant-ingressgateway
    namespace: gateway-system
  rules:
  - backendRefs:
    - name: toystore
      port: 80
EOF

(From now on and at anytime) Send requests to the application:

export INGRESS_IP=$(kubectl get gateway/kuadrant-ingressgateway -n gateway-system -o jsonpath='{.status.addresses[0].value}')

curl --resolve toystore.example.com:80:$INGRESS_IP http://toystore.example.com --write-out '%{http_code}\n' --silent --output /dev/null
curl --resolve other.example.com:80:$INGRESS_IP http://other.example.com --write-out '%{http_code}\n' --silent --output /dev/null

Deploy Kuadrant:

kubectl -n kuadrant-system apply -f - <<EOF
apiVersion: kuadrant.io/v1beta1
kind: Kuadrant
metadata:
  name: kuadrant
spec: {}
EOF

Create a gateway atomic default policy:

kubectl apply -n gateway-system -f - <<EOF
apiVersion: kuadrant.io/v1beta3
kind: RateLimitPolicy
metadata:
  name: gw-rlp
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: kuadrant-ingressgateway
  defaults:
    limits:
      "global":
        rates:
        - limit: 5
          duration: 10
          unit: second
        when:
        - selector: source.address
          operator: neq
          value: 127.0.0.1
EOF

Create a route policy:

kubectl apply -f - <<EOF
apiVersion: kuadrant.io/v1beta3
kind: RateLimitPolicy
metadata:
  name: route-rlp
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: toystore
  limits:
    "specific":
      rates:
      - limit: 3
        duration: 5
        unit: second
      - limit: 20
        duration: 1
        unit: minute
EOF

Modify the gateway policy to atomic override strategy:

kubectl apply -n gateway-system -f - <<EOF
apiVersion: kuadrant.io/v1beta3
kind: RateLimitPolicy
metadata:
  name: gw-rlp
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: kuadrant-ingressgateway
  overrides:
    limits:
      "global":
        rates:
        - limit: 5
          duration: 10
          unit: second
        when:
        - selector: source.address
          operator: neq
          value: 127.0.0.1
EOF

Modify the gateway policy to merge override strategy:

kubectl apply -n gateway-system -f - <<EOF
apiVersion: kuadrant.io/v1beta3
kind: RateLimitPolicy
metadata:
  name: gw-rlp
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: kuadrant-ingressgateway
  overrides:
    limits:
      "global":
        rates:
        - limit: 5
          duration: 10
          unit: second
        when:
        - selector: source.address
          operator: neq
          value: 127.0.0.1
    strategy: merge
EOF

Modify the route policy to define limits with the same name as limits defined by the gateway policy:

kubectl apply -f - <<EOF
apiVersion: kuadrant.io/v1beta3
kind: RateLimitPolicy
metadata:
  name: route-rlp
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: toystore
  limits:
    "global":
      rates:
      - limit: 3
        duration: 5
        unit: second
      - limit: 20
        duration: 1
        unit: minute
EOF

[codecov]

Codecov Report

Attention: Patch coverage is 5.84941% with 1513 lines in your changes missing coverage. Please review.

Project coverage is 28.14%. Comparing base (63f1d28) to head (5f1109a).
Report is 21 commits behind head on main.

Files with missing lines	Patch %	Lines
controllers/istio_rate_limit_cluster_reconciler.go	0.00%	155 Missing ⚠️
controllers/envoy_gateway_extension_reconciler.go	0.00%	142 Missing ⚠️
...ers/envoy_gateway_rate_limit_cluster_reconciler.go	0.00%	137 Missing ⚠️
controllers/ratelimitpolicy_status_updater.go	0.00%	136 Missing ⚠️
controllers/istio_extension_reconciler.go	0.00%	131 Missing ⚠️
controllers/ratelimit_workflow.go	26.66%	99 Missing ⚠️
api/v1beta3/ratelimitpolicy_types.go	9.25%	98 Missing ⚠️
pkg/wasm/utils.go	0.00%	97 Missing ⚠️
api/v1/merge_strategies.go	0.00%	79 Missing ⚠️
pkg/wasm/types.go	23.17%	63 Missing ⚠️
... and 12 more

❗ There is a different number of reports uploaded between BASE (63f1d28) and HEAD (5f1109a). Click for more details.

HEAD has 5 uploads less than BASE

Flag	BASE (63f1d28)	HEAD (5f1109a)
bare-k8s-integration	1	0
gatewayapi-integration	1	0
istio-integration	1	0
envoygateway-integration	1	0
controllers-integration	1	0

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #893       +/-   ##
===========================================
- Coverage   81.49%   28.14%   -53.36%     
===========================================
  Files         102      114       +12     
  Lines        7177     7763      +586     
===========================================
- Hits         5849     2185     -3664     
- Misses        898     5453     +4555     
+ Partials      430      125      -305     

Flag	Coverage Δ
bare-k8s-integration	?
controllers-integration	?
envoygateway-integration	?
gatewayapi-integration	?
istio-integration	?
unit	28.14% <5.84%> (-0.20%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
api/v1beta1 (u)	9.09% <ø> (-81.82%)	⬇️
api/v1beta2 (u)	45.66% <ø> (-40.95%)	⬇️
pkg/common (u)	79.66% <ø> (-8.48%)	⬇️
pkg/istio (u)	39.17% <3.22%> (-32.35%)	⬇️
pkg/log (u)	36.84% <ø> (-57.90%)	⬇️
pkg/reconcilers (u)	∅ <ø> (∅)
pkg/rlptools (u)	∅ <ø> (∅)
controllers (i)	13.36% <16.85%> (-69.70%)	⬇️

Files with missing lines	Coverage Δ
controllers/consoleplugin_reconciler.go	81.81% <100.00%> (ø)
pkg/envoygateway/mutators.go	0.00% <ø> (-36.12%)	⬇️
pkg/istio/mutators.go	0.00% <ø> (-50.00%)	⬇️
pkg/library/reconcilers/base_reconciler.go	29.72% <100.00%> (-36.06%)	⬇️
pkg/library/utils/hostname.go	100.00% <100.00%> (ø)
api/v1beta3/groupversion_info.go	0.00% <0.00%> (ø)
controllers/target_status_controller.go	0.00% <0.00%> (-90.29%)	⬇️
pkg/ratelimit/index.go	23.80% <50.00%> (ø)
pkg/library/kuadrant/errors.go	66.23% <5.00%> (-33.77%)	⬇️
controllers/state_of_the_world.go	3.69% <0.00%> (-93.55%)	⬇️
... and 17 more

... and 61 files with indirect coverage changes

[eguzki]

leave the policy status to me, it is WIP in another branch

[eguzki]

leave the policy status to me, it is WIP in another branch

Policy status belongs to this workflow. On you now.