POC: Vulnerability scanner based on Trivy file system scanner

[danielpacak]

Trivy can scan a container image by pulling the image from a remote registry, e.g.

trivy image --ignore-unfixed index.docker.io/library/nginx:1.16

or by directly scanning the container's local file system, e.g.

trivy fs --ignore-unfixed /

These two options allow us to choose between two different implementations of the vulnerabilityreport.Plugin.

The currently implemented plugin is based on the trivy image command (see Trivy Standalone and Trivy ClientServer), therefore, in this discussion we'll only focus on the local file system scan option.

Let's assume that there's the nginx Deployment in the poc-ns namespace.

---
apiVersion: v1
kind: Namespace
metadata:
  name: poc-ns
spec: { }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
  namespace: poc-ns
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      nodeName: kind-control-plane
      containers:
        - name: nginx
          image: nginx:1.16

To scan the nginx container of such Deployment, Starboard could create the following scan job in the starboard-operator namespace and observe it until it's completed or failed.

---
apiVersion: batch/v1
kind: Job
metadata:
  name: scan-nginx-with-trivy
  namespace: starboard-operator
spec:
  backoffLimit: 0
  template:
    spec:
      restartPolicy: Never
      # Explicit nodeName indicates our intention to schedule a scan pod
      # on the same node where the nginx workload is running.
      # This could also imply considering taints and tolerations and other
      # properties respected by K8s scheduler.
      nodeName: kind-control-plane
      volumes:
        - name: scan-volume
          emptyDir: { }
      initContainers:
        # The trivy-get-binary init container is used to copy out the trivy executable
        # binary from the upstream Trivy container image, i.e. aquasec/trivy:0.19.2,
        # to a shared emptyDir volume.
        - name: trivy-get-binary
          image: aquasec/trivy:0.19.2
          command:
            - cp
            - -v
            - /usr/local/bin/trivy
            - /var/starboard/trivy
          volumeMounts:
            - name: scan-volume
              mountPath: /var/starboard
         # The trivy-download-db container is using trivy executable binary
         # from the previous step to download Trivy vulnerability database
         # from GitHub releases page.
        - name: trivy-download-db
          image: aquasec/trivy:0.19.2
          command:
            - /var/starboard/trivy
            - --download-db-only
            - --cache-dir
            - /var/starboard/trivy-db
          volumeMounts:
            - name: scan-volume
              mountPath: /var/starboard
      containers:
        # The scan-nginx container is based on the container image that
        # we want to scan with Trivy. However, it has overwritten command (entrypoint)
        # to invoke trivy file system scan. The scan results are output to stdout
        # in JSON format so we can parse them and store as VulnerabiltyReport instance.
        - name: scan-nginx
          image: nginx:1.16
          # To scan image layers cached on K8s node without pulling
          # it from a remote registry.
          imagePullPolicy: Never
          command:
            - /var/starboard/trivy
            - --cache-dir
            - /var/starboard/trivy-db
            - fs
            - --format
            - json
            - --ignore-unfixed
            - /
          volumeMounts:
            - name: scan-volume
              mountPath: /var/starboard

$ kubectl get logs -n starboard-operator job/scan-nginx-with-trivy
2021-08-30T17:38:14.947Z	INFO	Detected OS: debian
2021-08-30T17:38:14.947Z	INFO	Detecting Debian vulnerabilities...
2021-08-30T17:38:14.953Z	INFO	Number of language-specific files: 2
2021-08-30T17:38:14.953Z	INFO	Detecting gobinary vulnerabilities...
2021-08-30T17:38:14.958Z	WARN	DEPRECATED: the current JSON schema is deprecated, check https://github.com/aquasecurity/trivy/discussions/1050 for more information.
[
  {
    "Target": "scan-nginx-with-trivy-tdpnb (debian 10.3)",
    "Class": "os-pkgs",
    "Type": "debian",
    "Vulnerabilities": [
      {
        "VulnerabilityID": "CVE-2020-27350",
        "PkgName": "apt",
        "InstalledVersion": "1.8.2",
        "FixedVersion": "1.8.2.2",
        "Layer": {
          "DiffID": "sha256:382dcd8abb00f344f52548954052edd513085623f3aff1a50aa17a52be513383"
        },
        "SeveritySource": "nvd",
        "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2020-27350",
        "Title": "apt: integer overflows and underflows while parsing .deb packages",
        "Description": "APT had several integer overflows and underflows while parsing .deb packages, aka GHSL-2020-168 GHSL-2020-169, in files apt-pkg/contrib/extracttar.cc, apt-pkg/deb/debfile.cc, and apt-pkg/contrib/arfile.cc. This issue affects: apt 1.2.32ubuntu0 versions prior to 1.2.32ubuntu0.2; 1.6.12ubuntu0 versions prior to 1.6.12ubuntu0.2; 2.0.2ubuntu0 versions prior to 2.0.2ubuntu0.2; 2.1.10ubuntu0 versions prior to 2.1.10ubuntu0.1;",
        "Severity": "MEDIUM",
        "CweIDs": [
          "CWE-190"
        ],
        "CVSS": {
          "nvd": {
            "V2Vector": "AV:L/AC:L/Au:N/C:P/I:P/A:P",
            "V3Vector": "CVSS:3.1/AV:L/AC:L/PR:H/UI:N/S:C/C:L/I:L/A:L",
            "V2Score": 4.6,
            "V3Score": 5.7
          },
          "redhat": {
            "V3Vector": "CVSS:3.1/AV:L/AC:L/PR:H/UI:N/S:C/C:L/I:L/A:L",
            "V3Score": 5.7
          }
        },
        "References": [
          "https://bugs.launchpad.net/bugs/1899193",
          "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-27350",
          "https://security.netapp.com/advisory/ntap-20210108-0005/",
          "https://ubuntu.com/security/notices/USN-4667-1",
          "https://ubuntu.com/security/notices/USN-4667-2",
          "https://usn.ubuntu.com/usn/usn-4667-1",
          "https://www.debian.org/security/2020/dsa-4808"
        ],
        "PublishedDate": "2020-12-10T04:15:00Z",
        "LastModifiedDate": "2021-01-08T12:15:00Z"
      },
[ shortened for readability ...]

Notes

1. This can be implemented as new mode of existing Trivy plugin. We already have Standalone and ClientServer mode and this can be called, e.g. FileSystem mode, or it can be enabled with a new config setting
2. If things go well, we'd like to extend this POC for workloads that run containers from private and managed registries and managed clusters, but the initial implementation should keep things simple. In particular:
   • Run scan jobs in the starboard-operator namespace
   • Ignore imagePullSecrets attached to workloads or service accounts
3. Keep the code on a dedicated branch / fork. The main goal is to play and learn from it.