docs: add FAQ entry clarifying that startup taints are necessary to a…

…void excess node provisioning

Signed-off-by: wmgroot <[email protected]>

website/content/en/preview/concepts/nodepools.md

@@ -72,7 +72,7 @@ spec:
           operator: In
           values: ["c", "m", "r"]
           # minValues here enforces the scheduler to consider at least that number of unique instance-category to schedule the pods.
-          # This field is ALPHA and can be dropped or replaced at any time 
+          # This field is ALPHA and can be dropped or replaced at any time
           minValues: 2
         - key: "karpenter.k8s.aws/instance-family"
           operator: In
@@ -556,6 +556,8 @@ In order for a pod to run on a node defined in this NodePool, it must tolerate `
 
 Per the Cilium [docs](https://docs.cilium.io/en/stable/installation/taints/#taint-effects), it's recommended to place a taint of `node.cilium.io/agent-not-ready=true:NoExecute` on nodes to allow Cilium to configure networking prior to other pods starting.  This can be accomplished via the use of Karpenter `startupTaints`.  These taints are placed on the node, but pods aren't required to tolerate these taints to be considered for provisioning.
 
+Failure to provide accurate `startupTaints` can result in Karpenter continually provisioning new nodes. When the new node joins and the startup taint that Karpenter is unaware of is added, Karpenter now considers the pending pod to be unschedulable to this node. Karpenter will attempt to provision yet another new node to schedule the pending pod.
+
 ```yaml
 apiVersion: karpenter.sh/v1beta1
 kind: NodePool

website/content/en/preview/contributing/documentation-updates.md

@@ -0,0 +1,11 @@
+---
+title: "Documentation Updates"
+linkTitle: "Documentation Updates"
+weight: 50
+description: >
+  Infomration helpful for contributing simple documentation updates.
+---
+
+- Documentation for https://karpenter.sh/docs/ is built under website/content/en/preview/.
+- Documentation updates should be made to the "preview" directory. Your changes will be promoted to website/content/en/docs/ by an automated process after the change has been merged.
+- Previews for your changes are built and available a few minutes after you push. Look for the "netlify Deploy Preview" link in a comment in your PR.

website/content/en/preview/faq.md

@@ -138,6 +138,25 @@ For more documentation on enabling IPv6 with the Amazon VPC CNI, see the [docs](
 Windows nodes do not support IPv6.
 {{% /alert %}}
 
+### Why do I see extra nodes get launched to schedule pending pods that remain empty and are later removed?
+
+You might have a daemonset, userData configuration, or some other workload that applies a taint after a node is provisioned. After the taint is applied, Karpenter will detect that the pod cannot be scheduled to this new node due to the added taint. As a result, Karpenter will provision yet another node. Typically, the original node has the taint removed and the pod schedules to it, leaving the extra new node unused and reaped by emptiness/consolidation. If the taint is not removed quickly enough, Karpenter may remove the original node before the pod can be scheduled via emptiness consolidation. This could result in an infinite loop of nodes being provisioned and consolidated without the pending pod ever scheduling.
+
+The solution is to configure [startupTaints]({{<ref "./concepts/nodepools/#cilium-startup-taint" >}}) to make Karpenter aware of any temporary taints that are needed to ensure that pods do not schedule on nodes that are not yet ready to receive them.
+
+Here's an example for Cilium's startup taint.
+```
+apiVersion: karpenter.sh/v1beta1
+kind: NodePool
+...
+spec:
+  template:
+    spec:
+      startupTaints:
+        - key: node.cilium.io/agent-not-ready
+          effect: NoSchedule
+```
+
 ## Scheduling
 
 ### When using preferred scheduling constraints, Karpenter launches the correct number of nodes at first.  Why do they then sometimes get consolidated immediately?