Network binding plugin: Address compute container resource overhead

Signed-off-by: Orel Misan <[email protected]>

design-proposals/network-binding-plugin/network-binding-plugin.md

@@ -281,6 +281,52 @@ This solution was selected since it provides the cluster admin more control in r
 
 For the alternative solutions please see [Appendix G](#appendix-g-alternatives-to-plugin-sidecar-container-resource-specification)
 
+##### Compute Container Resource Overhead
+
+For some plugins, an additional resource consumption can be expected from the virt-launcher pod compute container.
+For example, there could be need to execute an additional binary in the compute container.
+Since this binary has its own CPU and memory limits, they should be somehow accounted for.
+Another example could be increased resource consumption of the virt-stack resulting from using the plugin.
+
+Suggested solution:
+
+Additional API for compute container resource overhead:
+
+The network binding plugin API in the KubeVirt CR could receive an additional input field to specify the resource requirements overhead for the compute container:
+
+```yaml
+apiVersion: kubevirt.io/v1
+kind: KubeVirt
+metadata:
+  name: kubevirt
+  namespace: kubevirt
+spec:
+  configuration:
+    network:
+      binding:
+        mynetbindingplugin:
+          sidecarImage: quay.io/kubevirt/mynetbindingplugin
+          computeResourceOverhead:
+              cpu: 200m
+              memory: 20Mi
+```
+
+If specified, KubeVirt will add the resource overhead provided in `computeResourceOverhead` to the compute container.
+The resource overhead specification will only be added once, and not per NIC using the binding.
+
+Pros:
+- Cluster-wide definition of compute resource overhead per plugin.
+- Additional resources could be requested other than CPU and Memory.
+- The resource overhead specification is visible to cluster admins.
+
+Cons:
+- Requires an API change.
+- When upgrading KubeVirt / network binding plugin versions, the compute resource overhea specification might require adjustments.
+
+This solution was selected since it provides the cluster admin more control in regard to resource allocation.
+
+For the alternative solutions please see [Appendix H](#appendix-h-alternatives-to-compute-container-resource-overhead-specification)
+
 #### Configure Pod netns
 
 The CNI plugin has privileged access to the pod network namespace and
@@ -1262,3 +1308,21 @@ Cons:
 
 This solution provides flexibility for plugin authors while keeping the network binding plugin API in KubeVirt small.
 The requirement to maintain certificates for the webhook could be mitigated using tools such as [cert-manager](https://cert-manager.io/).
+
+# Appendix H:  Alternatives to compute container resource overhead specification
+
+1. Manually setting the VM's resources:
+
+The user can override KubeVirt's algorithms and set resource requirements.
+
+Pros:
+- Already implemented.
+
+Cons:
+- Error prune
+- The user does not take into account the overhead considerations KubeVirt takes when templating a virt-launcher pod.
+
+2. Mutating webhook
+
+Shipping a mutating webhook as part of the plugin's deliverables.
+The webhook will add the required resource overhead for the compute container.