Memory thrashing / nodes go Unready

[maximethebault]

What happened:
Using Karpenter to provision EKS AL2 nodes backed by this AMI.
When pods memory usage gets close to node's allocatable capacity, node goes into memory thrashing. Same symptoms & consequences as described on that issue: aws/karpenter-provider-aws#2129
We saw this issue happening on a r6id.xlarge instance type, with standard provisioner settings and standard usage. About 30 pods were scheduled on the node when this happened.

What you expected to happen:
kube-reserved / system-reserved / hard eviction correctly configured to avoid this situation.

How to reproduce it (as minimally and precisely as possible):
Let Karpenter provision a r6id.xlarge node for a pod with memory request close to allocatable capacity.
Make the memory used by this pod grow.

Anything else we need to know?:

Environment:

• AWS Region: eu-west-1
• Instance Type(s): r6id.xlarge
• EKS Platform version: eks.11
• Kubernetes version: 1.21
• AMI Version: all recent AMI release (saw some v20220824 as well as some v20221101)

[maximethebault]

Closed in favor of #1145