ControlGroups.md

Linux control groups inside Docker

Author: Tobit Flatscher (August 2021 - April 2022)

1. Control groups with PREEMPT

As already pointed out in realtime_basics/ControlGroups.md real-time groups require the kernel flag CONFIG_RT_GROUP_SCHED and therefore you will likely have to recompile your kernel. Follow the installation in that guide before continuing.

Be warned: Control groups are known to have a high amount of jitter and are mentioned here just for completeness. They are unlikely to be sufficient for real-time robotics.

1.1 Launching a real-time Docker with control groups

In order to run a real-time Docker you will first have to kill the Docker daemon if it is already running with

$ sudo systemctl stop docker
$ sudo systemctl stop docker.socket

and then re-open it assigning its control group a large time slice such as 950000:

$ sudo dockerd --cpu-rt-runtime=950000

The changes can also be made permanent by configuring the Docker daemon as described here and here.

Finally you can also launch the container with the real-time scheduler

$ sudo docker run -it --cpu-rt-runtime=950000 --ulimit rtprio=99 --cap-add=sys_nice ubuntu:focal

Similarly in a Docker-Compose file this can be achieved with (see here, here as well as here for more details):

privileged: true
cpu_rt_runtime: 950000
ulimits:
  rtprio: 99

The real-time runtime cpu.rt_runtime_us allocated for each group can be inspected in /sys/fs/cgroup/cpu,cpuacct. In case you have already allocated a large portion of real-time runtime to another cgroup this might result in the error message failed to write 95000 to cpu.rt_runtime_us: write /sys/fs/cgroup/cpu,cpuacct/system.slice/.../cpu.rt_runtime_us: invalid argument or similar (as discussed here and here). For more details on control groups in Linux see here and here. Now any process that is launched should be assigned to the corresponding real-time control group.