chore: upgrading to 0.30 api, creating v1 or v1beta1 VAP/VAPB #3472

JaydipGabani · 2024-08-02T18:15:41Z

Signed-off-by: Jaydip Gabani [email protected]
(cherry picked from commit 34275e7a564bf4f617bd232795d4426628cb7ece)

What this PR does / why we need it: In 1.30 if beta APIs are not enabled, VAP and VAPB generation will fail. This PR looks at available api version and creates VAP/VAPB

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #

Special notes for your reviewer:

Signed-off-by: Jaydip Gabani <[email protected]> (cherry picked from commit 34275e7a564bf4f617bd232795d4426628cb7ece)

maxsmythe

Mainly thoughts about how to handle multiple versions.

maxsmythe · 2024-08-02T18:21:43Z

config/crd/bases/mutations.gatekeeper.sh_assign.yaml

                          required:
                          - key
                          - operator
                          type: object
                        type: array
+                        x-kubernetes-list-type: atomic


Is adding this backwards compatible?

@maxsmythe I think so, this page says atomic can apply to any list. It is coming through bumping k8s api - https://github.com/kubernetes/apimachinery/blob/v0.30.3/pkg/apis/meta/v1/types.go#L1250

Backwards compatibility means that applying this tag will not change merge behavior.

In other words, we should be sure that kubectl apply logic prior to this tag being added and post this tag being added are unchanged.

Because this potentially affects backwards compatibility, we should be certain that this is the case. We should not rely on speculation. In other words, we should do more research than just inferring from a vague statement in the docs.

I found By default, Server-Side Apply treats custom resources as unstructured data. All keys are treated the same as struct fields, and all lists are considered atomic., here. So before this change, the list was being treated as atomic for patchStratagy=merge. wdyt?

How do I test for backward compatibility for this change?

I tested with deploying CRDs before this change and creating mutators, then I upgraded the CRDs with this change and applied mutators But I think this may be wrong way to test for backward compat, I am not sure.

I also see this If listType is missing, the API server interprets a patchStrategy=merge marker as a listType=map and the corresponding patchMergeKey marker as a listMapKey. on the same page, so now I am cofused.

I asked in slack - https://kubernetes.slack.com/archives/C0123CNN8F3/p1722991068800879, waiting for response.

I got the response - atomic is the default - https://kubernetes.slack.com/archives/C0123CNN8F3/p1722991724774759?thread_ts=1722991068.800879&cid=C0123CNN8F3.

@maxsmythe let me know if this resolves your concerns.

I am pasting the response for reference here about this change -

atomic is the default that was making it explicit to avoid linting errors about implicit list type

the containing labelselector struct is already annotated as +structType=atomic, which generators should turn into an atomic annotation on the parent selector field, so no list type annotations on the child fields should be relevant

Adding more context:

This x-kubernetes-list-type: atomic is getting added for list fields under metav1.labelSelector and metav1.namespaceSelector for mutators. This change is coming from here, where marker listType=atomic is getting added to matchExpression and values field. controller-runtime is looking at this markers and adding the field to manifets.

"By default, Server-Side Apply treats custom resources as unstructured data. All keys are treated the same as struct fields, and all lists are considered atomic." Sounds conclusive to me (from your first reply)

go.mod

maxsmythe · 2024-08-02T18:32:59Z

pkg/controller/constraint/constraint_controller.go

 					}
 					return reconcile.Result{}, err
 				}
 				if currentVapBinding == nil {
-					newVapBinding = transformedVapBinding.DeepCopy()
+					err = r.scheme.Convert(transformedVapBinding, newVapBinding, nil)


Will convert honor the GVK set on the unstructured object? It may be better/simpler to have newVapBinding be of Object type (the generic interface that matches all K8s objects, and set it to an empty instance of the appropriate versioned object via a function.

ex:

newVapBinding, err := vapBindingForVersion(gvk) // do err check r.scheme.Convert(transformedVapBinding, newVapBinding, nil) func vapBindingForVersion(gvk scheme.GroupVersion) (runtime.Object, error) { switch gvk.Version { case "v1": return &v1.VapBinding{} .... default: return nil, errors.New("unrecognized version") } }

Doing it this way will avoid uncertainties around how the type system handles unstructured and excess version juggling.

maxsmythe · 2024-08-02T18:36:44Z

pkg/controller/constraint/constraint_controller.go

 				} else {
 					newVapBinding = currentVapBinding.DeepCopy()
-					newVapBinding.Spec = transformedVapBinding.Spec
+					unstructuredObj, err := runtime.DefaultUnstructuredConverter.ToUnstructured(&transformedVapBinding.Spec)


Lets just separate this out to a separate function and use typecasting rather than resorting to unstructured.

maxsmythe · 2024-08-02T18:39:53Z

pkg/controller/constrainttemplate/constrainttemplate_controller.go

 			createErr := &v1beta1.CreateCRDError{Code: ErrCreateCode, Message: err.Error()}
 			status.Status.Errors = append(status.Status.Errors, createErr)
 			err := r.reportErrorOnCTStatus(ctx, ErrCreateCode, "Could not transform to vap object", status, err)
 			return reconcile.Result{}, err
 		}
 		if currentVap == nil {
-			newVap = transformedVap.DeepCopy()
+			err = r.scheme.Convert(transformedVap, newVap, nil)


Same concern about convert/unstructured. We can probably just use typed objects.

Using typed conversion.

ritazh · 2024-08-02T20:46:16Z

Makefile

@@ -172,8 +173,7 @@ e2e-bootstrap: e2e-dependencies
 	if [ $$(${GITHUB_WORKSPACE}/bin/kind get clusters) ]; then ${GITHUB_WORKSPACE}/bin/kind delete cluster; fi

 	# Create a new kind cluster
-	# TODO(ritazh): remove KIND_CLUSTER_FILE when vap feature is GA
-	if [ $$(echo $(KUBERNETES_VERSION) | cut -d'.' -f2) -lt 28 ]; then ${GITHUB_WORKSPACE}/bin/kind create cluster --image $(KIND_NODE_VERSION) --wait 5m; else ${GITHUB_WORKSPACE}/bin/kind create cluster --config $(KIND_CLUSTER_FILE) --image $(KIND_NODE_VERSION) --wait 5m; fi
+	if [ $$(echo $(KUBERNETES_VERSION) | cut -d'.' -f2) -lt 28 ]; then ${GITHUB_WORKSPACE}/bin/kind create cluster --image $(KIND_NODE_VERSION) --wait 5m; else ${GITHUB_WORKSPACE}/bin/kind create cluster --image $(KIND_NODE_VERSION) --wait 5m; fi


we still need the kind_cluster_file for versions before vap GAed in 1.30

Yip, adding the file back.

sozercan · 2024-08-02T20:54:11Z

go.mod

-go 1.21
+go 1.22.0
+
+toolchain go1.22.2


will this downgrade Go version? do we need GOTOOLCHAIN set to local for release?

removed toolchain go1.22.2

…ng a bug in IsVAPApiEnabled() Signed-off-by: Jaydip Gabani <[email protected]>

Signed-off-by: Jaydip Gabani <[email protected]>

ritazh · 2024-08-03T00:16:25Z

pkg/controller/constraint/constraint_controller.go

 	vapMux.Lock()
 	defer vapMux.Unlock()

 	if VapAPIEnabled != nil {
-		return *VapAPIEnabled
+		return *VapAPIEnabled, GroupVersion


Who sets GroupVersion? what if it's never set?

GroupVersion is set alongside VapAPIEnabled=true whenever VAP api with v1 or v1beta is found. If its never set, GroupVersion will be nil, but in that case *VapAPIEnabled will always be false.

since GroupVersion can be nil, please check for nil for the returned value from the caller function

Signed-off-by: Jaydip Gabani <[email protected]>

Signed-off-by: Jaydip Gabani <[email protected]> (cherry picked from commit 34275e7a564bf4f617bd232795d4426628cb7ece)

…ng a bug in IsVAPApiEnabled() Signed-off-by: Jaydip Gabani <[email protected]>

Signed-off-by: Jaydip Gabani <[email protected]>

codecov-commenter · 2024-08-05T20:43:54Z

Codecov Report

Attention: Patch coverage is 48.94895% with 170 lines in your changes missing coverage. Please review.

Project coverage is 48.07%. Comparing base (3350319) to head (5b3ffa0).
Report is 118 commits behind head on master.

Files	Patch %	Lines
pkg/controller/constraint/constraint_controller.go	0.00%	112 Missing ⚠️
...onstrainttemplate/constrainttemplate_controller.go	70.27%	27 Missing and 6 partials ⚠️
pkg/controller/pubsub/pubsub_config_controller.go	0.00%	15 Missing ⚠️
pkg/controller/mutators/core/adder.go	52.38%	10 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3350319) and HEAD (5b3ffa0). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (3350319) HEAD (5b3ffa0)

unittests 2 1

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3472      +/-   ##
==========================================
- Coverage   54.49%   48.07%   -6.43%     
==========================================
  Files         134      219      +85     
  Lines       12329    15165    +2836     
==========================================
+ Hits         6719     7290     +571     
- Misses       5116     7059    +1943     
- Partials      494      816     +322

Flag	Coverage Δ
unittests	`48.07% <48.94%> (-6.43%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Jaydip Gabani <[email protected]>

pkg/controller/constraint/constraint_controller.go

pkg/controller/constrainttemplate/constrainttemplate_controller.go

Signed-off-by: Jaydip Gabani <[email protected]>

maxsmythe · 2024-08-06T17:57:32Z

pkg/controller/constraint/constraint_controller.go

@@ -609,35 +628,128 @@ func (c *ConstraintsCache) reportTotalConstraints(ctx context.Context, reporter
 	}
 }

-func IsVapAPIEnabled() bool {
+func IsVapAPIEnabled() (bool, *schema.GroupVersion) {
 	vapMux.Lock()


We probably want to have an RWMutex here to enable parallelism in the future.

This is especially true because this function is being called by two different reconcilers (constraint/constraint template).

General pattern:

acquire read lock

check if VapAPIEnabled has been assigned a value, if so, return it

Otherwise, drop read lock and acquire a write lock

check if VapAPIEnabled has been assigned a value, if so, return it (this is necessary b/c a write could have happened in the time the lock was released)

continue as written

Added read lock

maxsmythe · 2024-08-06T17:59:47Z

pkg/controller/constraint/constraint_controller.go

+	}
+}
+
+func (r *ReconcileConstraint) getRunTimeVAPBinding(gvk *schema.GroupVersion, transformedVapBinding *admissionregistrationv1beta1.ValidatingAdmissionPolicyBinding, currentVapBinding client.Object) (client.Object, error) {


nit: this does not need to be a member function, it can be a standalone function

modified to standalone function.

maxsmythe · 2024-08-06T18:07:39Z

pkg/controller/constrainttemplate/constrainttemplate_controller.go

+	}
+}
+
+func (r *ReconcileConstraintTemplate) getRunTimeVAP(gvk *schema.GroupVersion, transformedVap *admissionregistrationv1beta1.ValidatingAdmissionPolicy, currentVap client.Object) (client.Object, error) {


nit: can be standalone function

modified to standalone function.

maxsmythe · 2024-08-06T18:08:47Z

pkg/controller/constrainttemplate/constrainttemplate_controller.go

+	return v1beta1VAP.DeepCopy(), nil
+}
+
+func v1beta1ToV1(v1beta1Obj *admissionregistrationv1beta1.ValidatingAdmissionPolicy) (*admissionregistrationv1.ValidatingAdmissionPolicy, error) {


Can we not use scheme.Convert here?

Direct conversion from v1beta1.ValidatingAdmissionPolicy to v1.ValidatingAdmissionPolicy using r.scheme.Convert results in error -
converting (v1beta1.ValidatingAdmissionPolicy) to (v1.ValidatingAdmissionPolicy): unknown conversion.

Oof, sounds like a bug in K8s code. Interim solution LGTM

@JaydipGabani do you want to open an issue for this on k/k? and add a comment to remove it after it's fixed in k/k?

Here is the issue - kubernetes/kubernetes#126582

maxsmythe · 2024-08-06T18:09:07Z

pkg/controller/constraint/constraint_controller.go

+	return v1beta1VAPBinding.DeepCopy(), nil
+}
+
+func v1beta1ToV1(v1beta1Obj *admissionregistrationv1beta1.ValidatingAdmissionPolicyBinding) (*admissionregistrationv1.ValidatingAdmissionPolicyBinding, error) {


Can we not use scheme.Convert() here?

Same happens with the binding as well, using r.scheme.Convert results in error in direct conversion.

Signed-off-by: Jaydip Gabani <[email protected]>

maxsmythe · 2024-08-06T22:03:21Z

pkg/controller/constraint/constraint_controller.go

@@ -629,6 +629,13 @@ func (c *ConstraintsCache) reportTotalConstraints(ctx context.Context, reporter
 }

 func IsVapAPIEnabled() (bool, *schema.GroupVersion) {
+	vapMux.RLock()
+	if VapAPIEnabled != nil {
+		vapMux.RUnlock()


For strict correctness, you should only access the variable while it's under a read lock. Suggest using intermediate variables.

Added local variables for strict correctness.

maxsmythe

LGTM after nit fix and after the backwards compatibility question is resolved.

Signed-off-by: Jaydip Gabani <[email protected]>

maxsmythe

Code LGTM, did not review tests, but defer to ritazh@/sozercan@ for those.

ritazh

LGTM

sozercan · 2024-08-07T22:42:23Z

pkg/controller/constraint/constraint_controller.go

 				transformedVapBinding, err := transform.ConstraintToBinding(instance, VAPEnforcementActions)
 				if err != nil {
 					status.Status.Errors = append(status.Status.Errors, constraintstatusv1beta1.Error{Message: err.Error()})
 					if err2 := r.writer.Update(ctx, status); err2 != nil {
-						log.Error(err2, "could not report transform vapbinding error status")
+						log.Error(err2, "could not report transform vapbinding error", "vapName", vapBindingName)


Suggested change

log.Error(err2, "could not report transform vapbinding error", "vapName", vapBindingName)

log.Error(err2, "could not report transform vapbinding error", "vapBindingName", vapBindingName)

sozercan · 2024-08-07T22:44:30Z

pkg/controller/constraint/constraint_controller.go

+				if err != nil {
+					status.Status.Errors = append(status.Status.Errors, constraintstatusv1beta1.Error{Message: err.Error()})
+					if err2 := r.writer.Update(ctx, status); err2 != nil {
+						log.Error(err2, "could not get VAP object with runtime group version", "vapName", vapBindingName)


Suggested change

log.Error(err2, "could not get VAP object with runtime group version", "vapName", vapBindingName)

log.Error(err2, "could not get VAP object with runtime group version", "vapBindingName", vapBindingName)

sozercan · 2024-08-07T22:51:20Z

pkg/controller/constraint/constraint_controller.go

+		return false, nil
+	}
+
+	groupVersion := schema.GroupVersion{Group: "admissionregistration.k8s.io", Version: "v1"}


Suggested change

groupVersion := schema.GroupVersion{Group: "admissionregistration.k8s.io", Version: "v1"}

groupVersion := schema.GroupVersion{Group: admissionregistrationv1beta1.GroupName, Version: "v1"}

sozercan · 2024-08-07T22:52:54Z

pkg/controller/constraint/constraint_controller.go

-		*VapAPIEnabled = false
-		return false
+
+	groupVersion = schema.GroupVersion{Group: "admissionregistration.k8s.io", Version: "v1beta1"}


Suggested change

groupVersion = schema.GroupVersion{Group: "admissionregistration.k8s.io", Version: "v1beta1"}

groupVersion = schema.GroupVersion{Group: admissionregistrationv1beta1.GroupName, Version: "v1beta1"}

sozercan · 2024-08-07T22:55:44Z

pkg/controller/constraint/constraint_controller.go

 	}
-	log.Info("IsVapAPIEnabled true")
+
+	log.Info("IsVapAPIEnabled", "error", err)


why are we logging an error as info?

sozercan · 2024-08-07T22:59:58Z

pkg/controller/constrainttemplate/constrainttemplate_controller_test.go

+		suffix := "VapShouldBeCreatedV1"
+
+		logger.Info("Running test: Vap should be created with v1")
+		constraint.GroupVersion = &schema.GroupVersion{Group: "admissionregistration.k8s.io", Version: "v1"}


Suggested change

constraint.GroupVersion = &schema.GroupVersion{Group: "admissionregistration.k8s.io", Version: "v1"}

constraint.GroupVersion = &schema.GroupVersion{Group: admissionregistrationv1beta1.GroupName, Version: "v1"}

hmm why didnt the test catch this?

Signed-off-by: Jaydip Gabani <[email protected]>

upgrading to 0.30 api, creating v1 or v1beta1 VAP/VAPB

4d3f669

Signed-off-by: Jaydip Gabani <[email protected]> (cherry picked from commit 34275e7a564bf4f617bd232795d4426628cb7ece)

JaydipGabani requested a review from a team as a code owner August 2, 2024 18:15

maxsmythe requested changes Aug 2, 2024

View reviewed changes

ritazh reviewed Aug 2, 2024

View reviewed changes

JaydipGabani mentioned this pull request Aug 2, 2024

chore: updating to use 0.30.3 k8s client open-policy-agent/frameworks#461

Merged

sozercan reviewed Aug 2, 2024

View reviewed changes

JaydipGabani added 2 commits August 2, 2024 23:52

adding config to enable beta api in kind cluster for 1.28, 1.29. fixi…

9298580

…ng a bug in IsVAPApiEnabled() Signed-off-by: Jaydip Gabani <[email protected]>

fixing lint

f43b6bf

Signed-off-by: Jaydip Gabani <[email protected]>

ritazh reviewed Aug 3, 2024

View reviewed changes

JaydipGabani added 9 commits August 5, 2024 04:01

adding tests

9f58850

Signed-off-by: Jaydip Gabani <[email protected]>

upgrading to 0.30 api, creating v1 or v1beta1 VAP/VAPB

ef94027

Signed-off-by: Jaydip Gabani <[email protected]> (cherry picked from commit 34275e7a564bf4f617bd232795d4426628cb7ece)

adding config to enable beta api in kind cluster for 1.28, 1.29. fixi…

f6b9ebe

…ng a bug in IsVAPApiEnabled() Signed-off-by: Jaydip Gabani <[email protected]>

fixing lint

c5793ea

Signed-off-by: Jaydip Gabani <[email protected]>

adding tests

3a0eede

Signed-off-by: Jaydip Gabani <[email protected]>

using runtime.Object for VAP/VAPB versions

bb19964

Signed-off-by: Jaydip Gabani <[email protected]>

merging with master

f8791e6

Signed-off-by: Jaydip Gabani <[email protected]>

converting v1beta1 to v1 without unstructured

1dff1e5

Signed-off-by: Jaydip Gabani <[email protected]>

removing toolchain from go.mod

de48c99

Signed-off-by: Jaydip Gabani <[email protected]>

JaydipGabani force-pushed the k130 branch from bb0324d to de48c99 Compare August 5, 2024 20:32

JaydipGabani force-pushed the k130 branch from 6784609 to de48c99 Compare August 5, 2024 21:44

JaydipGabani requested review from sozercan, maxsmythe and ritazh August 5, 2024 22:06

removing lib file

1e26786

Signed-off-by: Jaydip Gabani <[email protected]>

ritazh reviewed Aug 6, 2024

View reviewed changes

pkg/controller/constraint/constraint_controller.go Outdated Show resolved Hide resolved

ritazh reviewed Aug 6, 2024

View reviewed changes

pkg/controller/constraint/constraint_controller.go Outdated Show resolved Hide resolved

ritazh reviewed Aug 6, 2024

View reviewed changes

pkg/controller/constraint/constraint_controller.go Outdated Show resolved Hide resolved

ritazh reviewed Aug 6, 2024

View reviewed changes

pkg/controller/constrainttemplate/constrainttemplate_controller.go Outdated Show resolved Hide resolved

ritazh reviewed Aug 6, 2024

View reviewed changes

pkg/controller/constrainttemplate/constrainttemplate_controller.go Outdated Show resolved Hide resolved

checking for VAP API availability only when generation is on

ed5c510

Signed-off-by: Jaydip Gabani <[email protected]>

JaydipGabani requested a review from ritazh August 6, 2024 03:32

sozercan added this to the v3.17.0 milestone Aug 6, 2024

maxsmythe requested changes Aug 6, 2024

View reviewed changes

JaydipGabani added 2 commits August 6, 2024 18:26

addressing nits

fe4998e

Signed-off-by: Jaydip Gabani <[email protected]>

adding read lock while checking for vap api availability

3a77ea3

Signed-off-by: Jaydip Gabani <[email protected]>

JaydipGabani requested a review from maxsmythe August 6, 2024 20:09

Merge branch 'master' into k130

a63f998

maxsmythe reviewed Aug 6, 2024

View reviewed changes

JaydipGabani added 4 commits August 6, 2024 23:12

adding label-selector in mutation for backward comp check

6cb62fa

Signed-off-by: Jaydip Gabani <[email protected]>

Merge branch 'k130' of github.com:JaydipGabani/gatekeeper into k130

7e39997

using interim variables for read lock

f3a4977

Signed-off-by: Jaydip Gabani <[email protected]>

adding todo to use r.scheme.convert

2f0b758

Signed-off-by: Jaydip Gabani <[email protected]>

JaydipGabani requested a review from maxsmythe August 7, 2024 17:28

maxsmythe approved these changes Aug 7, 2024

View reviewed changes

ritazh approved these changes Aug 7, 2024

View reviewed changes

sozercan reviewed Aug 7, 2024

View reviewed changes

addressing nits

5b3ffa0

Signed-off-by: Jaydip Gabani <[email protected]>

sozercan approved these changes Aug 7, 2024

View reviewed changes

JaydipGabani merged commit 36b75d7 into open-policy-agent:master Aug 8, 2024
21 checks passed

JaydipGabani deleted the k130 branch August 8, 2024 00:27

sozercan mentioned this pull request Aug 9, 2024

chore: Update to use k8s client libs 1.30.0 #3389

Closed

	log.Error(err2, "could not report transform vapbinding error", "vapName", vapBindingName)
	log.Error(err2, "could not report transform vapbinding error", "vapBindingName", vapBindingName)

	log.Error(err2, "could not get VAP object with runtime group version", "vapName", vapBindingName)
	log.Error(err2, "could not get VAP object with runtime group version", "vapBindingName", vapBindingName)

	groupVersion := schema.GroupVersion{Group: "admissionregistration.k8s.io", Version: "v1"}
	groupVersion := schema.GroupVersion{Group: admissionregistrationv1beta1.GroupName, Version: "v1"}

	groupVersion = schema.GroupVersion{Group: "admissionregistration.k8s.io", Version: "v1beta1"}
	groupVersion = schema.GroupVersion{Group: admissionregistrationv1beta1.GroupName, Version: "v1beta1"}

	constraint.GroupVersion = &schema.GroupVersion{Group: "admissionregistration.k8s.io", Version: "v1"}
	constraint.GroupVersion = &schema.GroupVersion{Group: admissionregistrationv1beta1.GroupName, Version: "v1"}

chore: upgrading to 0.30 api, creating v1 or v1beta1 VAP/VAPB #3472

chore: upgrading to 0.30 api, creating v1 or v1beta1 VAP/VAPB #3472

Conversation

JaydipGabani commented Aug 2, 2024

maxsmythe left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maxsmythe Aug 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 5, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ritazh Aug 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maxsmythe left a comment

Choose a reason for hiding this comment

maxsmythe left a comment

Choose a reason for hiding this comment

ritazh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maxsmythe Aug 6, 2024 •

edited

Loading

codecov-commenter commented Aug 5, 2024 •

edited

Loading

ritazh Aug 7, 2024 •

edited

Loading