NetworkServicesGRPCRoute objects fail to sync with "Location" must be set after mutation

[yzhaoa]

Checklist

• I did not find a related open issue.
• I did not find a solution in the troubleshooting guide: (https://cloud.google.com/config-connector/docs/troubleshooting)
• If this issue is time-sensitive, I have submitted a corresponding issue with GCP support.

Bug Description

We have about 10-12 NetworkServicesGRPCRoute objects in our config that we may update at a time. Invariably when we update the rules of these NetworkServicesGRPCRoute objects, e.g. to retarget grpc routes to a new ComputeBackendService, ~4-5 fail to update with the following status:

Update call failed: error generating the diffs from the desired state: "Location" must be set

Additional Diagnostic Information

On all cases of failed update, I have identified the following situation in the {.metadata.managedFields} section of the object:

metadata:
  managedFields:
    ...
    - apiVersion: networkservices.cnrm.cloud.google.com/v1beta1
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            f:kubectl.kubernetes.io/last-applied-configuration: {}
          f:labels:
            f:app.kubernetes.io/version: {}
        f:spec:
          f:rules: {}
      manager: kubectl-client-side-apply
      operation: Update
      subresource: status
      time: "..."

A properly functioning object always have the following instead:

metadata:
  managedFields:
  ...
  - apiVersion: networkservices.cnrm.cloud.google.com/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
        f:labels:
          .: {}
          f:app.kubernetes.io/managed-by: {}
          f:app.kubernetes.io/version: {}
      f:spec:
        .: {}
        f:hostnames: {}
        f:location: {}
        f:meshes: {}
        f:resourceID: {}
        f:rules: {}
    manager: kubectl-client-side-apply
    operation: Update
    time: "..."

Note how every field under {.spec} are missing from the field mask except the rules field, which was mutated.

Manually fixing the field manager annotations by forcing it to look like the latter fixes the error.

Kubernetes Cluster Version

Client Version: v1.30.8-dispatcher Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.30.8-gke.1051000

Config Connector Version

1.116.0

Config Connector Mode

namespaced mode (default)

Log Output

No response

Steps to reproduce the issue

1. Define 10 NetworkServicesGRPCRoute and apply them to a config-connector-enabled namespace.
2. Wait for sync.
3. Update all 10 routes to point to a different backend.
4. Wait for sync.
5. Sync fails with errors on some routes.

YAML snippets

apiVersion: networkservices.cnrm.cloud.google.com/v1beta1
kind: NetworkServicesGRPCRoute
metadata:
  name: <something>-service-gke-route
  namespace: internal-namespace
spec:
  hostnames:
    - <something>-service:80
  location: global
  meshes:
    - name: grpcmesh
  projectRef:
    external: <project id>
  resourceID: <something>-service-gke-route
  rules:
    - action:
        destinations:
          - serviceRef:
              name: <some backend>