cis-1.3.1: 5.10 and 5.11 (possibly more checks) don't work properly

[marcindulak]

Observation

The problem seems to be due to the format of the output returned by the audit command, which contains the container Id.
For example

docker ps --quiet --all | xargs docker inspect --format '{{ .Id }}:Memory={{ .HostConfig.Memory }}'
87d3272a4a42fdeb534935bccb9b934f13761fc7e8ba82e233de581213d05359:Memory=268435456

is considered a FAIL by

docker run --memory=256m hello-world:latest
sudo docker-bench --benchmark cis-1.3.1 --include-test-output --config-dir /etc/docker-bench/cfg | grep "] 5.10" -A 1

[FAIL] 5.10 Ensure that the memory usage for container is limited (Automated)
         87d3272a4a42fdeb534935bccb9b934f13761fc7e8ba82e233de581213d05359:Memory=268435456

Possible solution

It seems like the problem can be resolved by using a regex

    tests:
      test_items:
      - flag: ".*Memory"

instead of

docker-bench/cfg/cis-1.3.1/definitions.yaml

Line 1313 in b6e798b

- flag: "Memory"

Other affected checks

docker-bench/cfg/cis-1.3.1/definitions.yaml

Line 1333 in b6e798b

- id: 5.11

is also affected, possibly more checks are affected.

System details

cat /etc/*release
NAME="openSUSE Leap"
VERSION="15.3"
ID="opensuse-leap"
ID_LIKE="suse opensuse"
VERSION_ID="15.3"
PRETTY_NAME="openSUSE Leap 15.3"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:opensuse:leap:15.3"
BUG_REPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org/"

docker version
Client:
 Version:           20.10.6-ce
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        8728dd246c3a
 Built:             Thu Apr 15 12:00:00 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.6-ce
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       8728dd246c3a
  Built:            Thu Apr 15 12:00:00 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.1.5_catatonit
  GitCommit:        

docker-bench https://github.com/aquasecurity/docker-bench/releases/tag/v0.5.0 docker-bench_0.5.0_linux_amd64.rpm