doc: update balloons documentation to the CRD era

Signed-off-by: Antti Kervinen <[email protected]>

docs/resource-policy/policy/balloons.md

@@ -61,20 +61,26 @@ more details on managing the configuration.
 
 Balloons policy parameters:
 
-- `PinCPU` controls pinning a container to CPUs of its balloon. The
+- `reservedResources`:
+  - `cpu` specifies cpuset or number of CPUs in the special `reserved`
+    balloon. By default all containers in the `kube-system` namespace
+    are assigned to the reserved balloon. Examples: `cpu: cpuset:0,48`
+    uses two logical CPUs: cpu0 and cpu48. `cpu: 2000m` uses
+    any two CPUs.
+- `pinCPU` controls pinning a container to CPUs of its balloon. The
   default is `true`: the container cannot use other CPUs.
-- `PinMemory` controls pinning a container to the memories that are
+- `pinMemory` controls pinning a container to the memories that are
   closest to the CPUs of its balloon. The default is `true`: allow
   using memory only from the closest NUMA nodes. Warning: this may
   cause kernel to kill workloads due to out-of-memory error when
   closest NUMA nodes do not have enough memory. In this situation
   consider switching this option `false`.
-- `IdleCPUClass` specifies the CPU class of those CPUs that do not
+- `idleCPUClass` specifies the CPU class of those CPUs that do not
   belong to any balloon.
-- `ReservedPoolNamespaces` is a list of namespaces (wildcards allowed)
+- `reservedPoolNamespaces` is a list of namespaces (wildcards allowed)
   that are assigned to the special reserved balloon, that is, will run
   on reserved CPUs. This always includes the `kube-system` namespace.
-- `AllocatorTopologyBalancing` affects selecting CPUs for new
+- `allocatorTopologyBalancing` affects selecting CPUs for new
   balloons. If `true`, new balloons are created using CPUs on
   NUMA/die/package with most free CPUs, that is, balloons are spread
   across the hardware topology. This helps inflating balloons within
@@ -83,7 +89,7 @@ Balloons policy parameters:
   pack new balloons tightly into the same NUMAs/dies/packages. This
   helps keeping large portions of hardware idle and entering into deep
   power saving states.
-- `PreferSpreadOnPhysicalCores` prefers allocating logical CPUs
+- `preferSpreadOnPhysicalCores` prefers allocating logical CPUs
   (possibly hyperthreads) for a balloon from separate physical CPU
   cores. This prevents workloads in the balloon from interfering with
   themselves as they do not compete on the resources of the same CPU
@@ -93,46 +99,47 @@ Balloons policy parameters:
   value set here is the default for all balloon types, but it can be
   overridden with the balloon type specific setting with the same
   name.
-- `BalloonTypes` is a list of balloon type definitions. Each type can
+- `balloonTypes` is a list of balloon type definitions. Each type can
   be configured with the following parameters:
-  - `Name` of the balloon type. This is used in pod annotations to
+  - `name` of the balloon type. This is used in pod annotations to
     assign containers to balloons of this type.
-  - `Namespaces` is a list of namespaces (wildcards allowed) whose
+  - `namespaces` is a list of namespaces (wildcards allowed) whose
     pods should be assigned to this balloon type, unless overridden by
     pod annotations.
-  - `MinBalloons` is the minimum number of balloons of this type that
+  - `minBalloons` is the minimum number of balloons of this type that
     is always present, even if the balloons would not have any
     containers. The default is 0: if a balloon has no containers, it
     can be destroyed.
-  - `MaxBalloons` is the maximum number of balloons of this type that
+  - `maxBalloons` is the maximum number of balloons of this type that
     is allowed to co-exist. The default is 0: creating new balloons is
     not limited by the number of existing balloons.
-  - `MaxCPUs` specifies the maximum number of CPUs in any balloon of
+  - `maxCPUs` specifies the maximum number of CPUs in any balloon of
     this type. Balloons will not be inflated larger than this. 0 means
     unlimited.
-  - `MinCPUs` specifies the minimum number of CPUs in any balloon of
+  - `minCPUs` specifies the minimum number of CPUs in any balloon of
     this type. When a balloon is created or deflated, it will always
     have at least this many CPUs, even if containers in the balloon
     request less.
-  - `CpuClass` specifies the name of the CPU class according to which
-    CPUs of balloons are configured.
-  - `PreferCloseToDevices`: prefer creating new balloons close to
+  - `cpuClass` specifies the name of the CPU class according to which
+    CPUs of balloons are configured. Class properties are defined in
+    separate `cpu.classes` objects, see below.
+  - `preferCloseToDevices`: prefer creating new balloons close to
     listed devices. List of strings
-  - `PreferSpreadingPods`: if `true`, containers of the same pod
+  - `preferSpreadingPods`: if `true`, containers of the same pod
     should be spread to different balloons of this type. The default
     is `false`: prefer placing containers of the same pod to the same
     balloon(s).
-  - `PreferPerNamespaceBalloon`: if `true`, containers in the same
+  - `preferPerNamespaceBalloon`: if `true`, containers in the same
     namespace will be placed in the same balloon(s). On the other
     hand, containers in different namespaces are preferrably placed in
     different balloons. The default is `false`: namespace has no
     effect on choosing the balloon of this type.
-  - `PreferNewBalloons`: if `true`, prefer creating new balloons over
+  - `preferNewBalloons`: if `true`, prefer creating new balloons over
     placing containers to existing balloons. This results in
     preferring exclusive CPUs, as long as there are enough free
     CPUs. The default is `false`: prefer filling and inflating
     existing balloons over creating new ones.
-  - `ShareIdleCPUsInSame`: Whenever the number of or sizes of balloons
+  - `shareIdleCPUsInSame`: Whenever the number of or sizes of balloons
     change, idle CPUs (that do not belong to any balloon) are reshared
     as extra CPUs to workloads in balloons with this option. The value
     sets locality of allowed extra CPUs that will be common to these
@@ -145,70 +152,87 @@ Balloons policy parameters:
     - `numa`: ...in the same numa node(s) as the balloon.
     - `core`: ...allowed to use idle CPU threads in the same cores with
       the balloon.
-  - `PreferSpreadOnPhysicalCores` overrides the policy level option
+  - `preferSpreadOnPhysicalCores` overrides the policy level option
     with the same name in the scope of this balloon type.
-  - `PreferCloseToDevices` prefers creating new balloons close to
+  - `preferCloseToDevices` prefers creating new balloons close to
     listed devices. If all preferences cannot be fulfilled, preference
     to first devices in the list override preferences to devices after
     them. Adding this preference to any balloon type automatically
     adds corresponding anti-affinity to other balloon types that do
     not prefer to be close to the same device: they prefer being
     created away from the device. Example:
     ```
-    PreferCloseToDevices:
+    preferCloseToDevices:
       - /sys/class/net/eth0
       - /sys/class/block/sda
     ```
-  - `AllocatorPriority` (0: High, 1: Normal, 2: Low, 3: None). CPU
+  - `allocatorPriority` (0: High, 1: Normal, 2: Low, 3: None). CPU
     allocator parameter, used when creating new or resizing existing
     balloons. If there are balloon types with pre-created balloons
-    (`MinBalloons` > 0), balloons of the type with the highest
-    `AllocatorPriority` are created first.
-
-Related configuration parameters:
-
-- `policy.ReservedResources.CPU` specifies the (number of) CPUs in the
-  special `reserved` balloon. By default all containers in the
-  `kube-system` namespace are assigned to the reserved balloon.
-- `cpu.classes` defines CPU classes and their parameters (such as
-  `minFreq`, `maxFreq`, `uncoreMinFreq` and `uncoreMaxFreq`).
+    (`minBalloons` > 0), balloons of the type with the highest
+    `allocatorPriority` are created first.
+- `control.cpu.classes`: defines CPU classes and their
+    properties. Class names are keys followed by properties:
+    - `minFreq` minimum frequency for CPUs in this class (kHz).
+    - `maxFreq` maximum frequency for CPUs in this class (kHz).
+    - `uncoreMinFreq` minimum uncore frequency for CPUs in this
+      class (kHz).  If there are differences in `uncoreMinFreq`s in
+      CPUs within the same uncore frequency zone, the maximum value
+      of all `uncoreMinFreq`s is used.
+    - `uncoreMaxFreq` maximum uncore frequency for CPUs in this
+      class (kHz).
+- `instrumentation`: configures interface for runtime instrumentation.
+  - `httpEndpoint`: the address the HTTP server listens on. Example:
+    `:8891`.
+  - `prometheusExport`: if set to True, balloon instances, their CPUs
+     and assigned workloads are readable through `/metrics` from the
+     httpEndpoint.
+  - `reportPeriod`: `/metrics` aggregation interval.
 
 ### Example
 
-Example configuration that runs all pods in balloons of 1-4 CPUs.
+Example configuration that runs all pods in balloons of 1-4
+CPUs. Instrumentation enables reading CPUs and workloads in balloons
+from `http://localhost:8891/metrics`.
 
 ```yaml
-  ReservedResources:
-    CPU: 1
-  balloons:
-    PinCPU: true
-    PinMemory: true
-    IdleCPUClass: lowpower
-    BalloonTypes:
-      - Name: "quad"
-        MinCpus: 1
-        MaxCPUs: 4
-        CPUClass: dynamic
-        Namespaces:
-          - "*"
-cpu:
-  classes:
-    lowpower:
-      minFreq: 800
-      maxFreq: 800
-    dynamic:
-      minFreq: 800
-      maxFreq: 3600
-    turbo:
-      minFreq: 3000
-      maxFreq: 3600
-      uncoreMinFreq: 2000
-      uncoreMaxFreq: 2400
+apiVersion: config.nri/v1alpha1
+kind: BalloonsPolicy
+metadata:
+  name: default
+  namespace: kube-system
+spec:
+  reservedResources:
+    cpu: 1000m
+  pinCPU: true
+  pinMemory: true
+  allocatorTopologyBalancing: true
+  idleCPUClass: lowpower
+  balloonTypes:
+    - name: "quad"
+      maxCPUs: 4
+      cpuClass: dynamic
+      namespaces:
+        - "*"
+  control:
+    cpu:
+      classes:
+        lowpower:
+          minFreq: 800000
+          maxFreq: 800000
+        dynamic:
+          minFreq: 800000
+          maxFreq: 3600000
+        turbo:
+          minFreq: 3000000
+          maxFreq: 3600000
+          uncoreMinFreq: 2000000
+          uncoreMaxFreq: 2400000
+  instrumentation:
+    httpEndpoint: :8891
+    prometheusExport: true
 ```
 
-See the [sample configuration custom resource](/sample-configs/balloons-config.yaml) for a
-more complete example.
-
 ## Assigning a Container to a Balloon
 
 The balloon type of a container can be defined in pod annotations. In