Initial CLI scanner commit (#18) (#19)

* Initial CLI scanner commit (#18) * Initial CLI scanner commit * Update install.md * Using bash not sh * Update for CVSS and temp removal of description * Update to revert to v1Beta1 public APIs. Leaving in most V2 code commented out until agree to commit to main * Update to revert to v1Beta1 public APIs. Leaving in most V2 code commented out until agree to commit to main * Update to revert to v1Beta1 public APIs. Leaving in most V2 code commented out until agree to commit to main * Removing v2 code altogether * Update README.md * Update for SaaS * Update install.md * Readme Updates. * Readme Updates. * Mock Update(s) * Test updates and removal of backend tests that are no longer needed * Update ci.yaml * Update image digest for tests * Set original test image * Update image digest for tests * Pump up golang builder version in Dockerfile * Updated tests, commented out backend adapter (#20) * Dev new engine (#21) * Updated tests, commented out backend adapter * Update of scanner job logic to handle the sysdig-cli-scanner container * Dev new engine (#22) * Updated tests, commented out backend adapter * Update of scanner job logic to handle the sysdig-cli-scanner container * Test update to cater for new job spec --------- Co-authored-by: Aaron Miles <[email protected]>
sysdiglabs · May 27, 2024 · e782f3f · e782f3f
1 parent 7448ee5
commit e782f3f
Show file tree

Hide file tree

Showing 24 changed files with 1,087 additions and 817 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -44,7 +44,7 @@ jobs:
       run: make test
       env:
         SECURE_API_TOKEN: ${{ secrets.KUBELAB_SECURE_API_TOKEN }}
-        SECURE_URL: https://secure.sysdig.com
+        SECURE_URL: ${{ vars.SECURE_URL }}
 
   docker:
     name: Build Docker Image

diff --git a/README.md b/README.md
@@ -9,46 +9,20 @@ The Sysdig Secure Harbor Scanner Adapter enables Harbor to use Sysdig Secure sca
 This adapter also provides a service that translates the Harbor scanning API requests into Sysdig Secure API calls, allowing Harbor to retrieve vulnerability reports and additional information from the scanning adapter. This information will be presented in the Harbor UI, transparently for the user.
 
 ## Getting Started
-
 You can follow a [detailed guide to deploy the Scanner Adapter](docs/install.md).
 
-## Inline and Backend Scanning
-
-This scanning adapter has two operation modes:
-* Backend Scanning: Image scanning happens in the Sysdig Secure Backend
-* Inline Scanning: Image scanning happens in the infrastructure where Harbor is hosted
-
-### Backend Scanning
-
-This is the default mode. The Sysdig Harbor adapter will forward the container image path to the Sysdig Secure backend (either SaaS or Onprem), for example `docker.io/alpine:latest`. The backend will use this path to retrieve and scan the container image, providing the results back to the Sysdig Harbor adapter.
-
-PRO:
-* Easier to install
-
-CON:
-* Sysdig Secure Backend needs to have network visibility in order to fetch images from Harbor
-
-### Inline Scanning
-
-Using inline scanning, the scanning operation itself will be triggered and performed on your own infrastructure. It spawns a Kubernetes job when a new image is pushed, this job will communicate **only** the container metadata to the Sysdig Secure Backend, which will perform the evaluation based on the configured image [scanning policies](https://docs.sysdig.com/en/manage-scanning-policies.html).
-
-PRO:
-* No need to configure registry credentials in the Sysdig Secure Backend
-* No need to expose your registry externally, so it can be reached by Sysdig Secure (see CON in the section above)
-* Image contents are never transmitted outside the pipeline, just the image metadata
-
-CON:
-* The job performing the inline scanning needs to have access to the host-local Docker daemon
+### CLI Scanning
+Using CLI scanning, the scanning operation itself will be triggered and performed on your own infrastructure. It spawns a Kubernetes job when a new image is pushed, this job will communicate **only** the container metadata to the Sysdig Secure Backend, which will perform the evaluation based on the configured image [scanning policies](https://docs.sysdig.com/en/manage-scanning-policies.html).
 
 ## Configuration
 
 Configuration of the adapter is done via environment variables at startup.
 
-| Name               | Default | Description                                                               |
-| ---                | ---     | ---                                                                       |
-| `SECURE_URL`       | ` `     | Sysdig Secure URL                                                         |
-| `SECURE_API_TOKEN` | ` `     | Sysdig Secure API Token                                                   |
-| `INLINE_SCANNING`  | ` `     | Enable Inline Scanning instead of Backend                                 |
-| `NAMESPACE_NAME`   | ` `     | Namespace where Inline Scanning will spawn jobs                           |
-| `CONFIGMAP_NAME`   | ` `     | ConfigMap name where Harbor Certificate is available                      |
-| `SECRET_NAME`      | ` `     | Secret name where Sysdig Secure API Token and Robot Account are available |
+| Name              | Default                     | Description                                                               |
+|-------------------|-----------------------------| ---                                                                       |
+| `SECURE_URL`      | `https://secure.sysdig.com` | Sysdig Secure URL                                                         |
+| `SECURE_API_TOKEN` | ` `                         | Sysdig Secure API Token                                                   |
+| `CLI_SCANNING`    | ` `                         | Enable CLI Scanning instead of Backend                                 |
+| `NAMESPACE_NAME`  | ` `                         | Namespace where CLI Scanning will spawn jobs                           |
+| `CONFIGMAP_NAME`  | ` `                         | ConfigMap name where Harbor Certificate is available                      |
+| `SECRET_NAME`     | ` `                         | Secret name where Sysdig Secure API Token and Robot Account are available |
diff --git a/build/Dockerfile b/build/Dockerfile
@@ -1,4 +1,4 @@
-FROM golang:1.16 as builder
+FROM golang:1.22.1 as builder
 WORKDIR /harbor-scanner-sysdig-secure
 COPY go.mod go.sum ./
 COPY . .

diff --git a/cmd/harbor-scanner-sysdig-secure/main.go b/cmd/harbor-scanner-sysdig-secure/main.go
@@ -53,11 +53,12 @@ func configure() error {
 	pflag.String("secure_api_token", "", "Sysdig Secure API Token")
 	pflag.String("secure_url", "https://secure.sysdig.com", "Sysdig Secure URL Endpoint")
 	pflag.Bool("verify_ssl", true, "Verify SSL when connecting to Sysdig Secure URL Endpoint")
-	pflag.Bool("inline_scanning", false, "Use Inline Scanning Adapter")
+	pflag.Bool("cli_scanning", false, "Use Sysdig-Cli-Scanner Scanning Adapter")
 	pflag.Bool("async_mode", false, "Use Async-Mode to perform reports retrieval")
 	pflag.String("namespace_name", "", "Namespace where inline scanning jobs are spawned")
 	pflag.String("secret_name", "", "Secret which keeps the inline scanning secrets ")
-	pflag.String("inline_scanning_extra_params", "", "Extra parameters to provide to inline-scanner")
+	pflag.String("cli_scanning_extra_params", "", "Extra parameters to provide to cli-scanner")
+	pflag.String("cli_scanner_image", "", "Extra parameters to provide to cli-scanner")
 
 	pflag.VisitAll(func(flag *pflag.Flag) { viper.BindPFlag(flag.Name, flag) })
 
@@ -67,8 +68,8 @@ func configure() error {
 		return errors.New("secure_api_token is required")
 	}
 
-	if viper.GetBool("inline_scanning") && (viper.Get("namespace_name") == "" || viper.Get("secret_name") == "") {
-		return errors.New("namespace_name and secret_name are required when running inline scanning")
+	if viper.GetBool("cli_scanning") && (viper.Get("namespace_name") == "" || viper.Get("secret_name") == "") {
+		return errors.New("namespace_name and secret_name are required when running sysdig-cli-scanner")
 	}
 
 	return nil
@@ -77,8 +78,8 @@ func configure() error {
 func getAdapter() scanner.Adapter {
 	client := secure.NewClient(viper.GetString("secure_api_token"), viper.GetString("secure_url"), viper.GetBool("verify_ssl"))
 
-	if viper.GetBool("inline_scanning") {
-		log.Info("Using inline-scanning adapter")
+	if viper.GetBool("cli_scanning") {
+		log.Info("Using cli-scanner adapter")
 		config, err := rest.InClusterConfig()
 		if err != nil {
 			log.Fatal(err)
@@ -95,11 +96,11 @@ func getAdapter() scanner.Adapter {
 			viper.GetString("secure_url"),
 			viper.GetString("namespace_name"),
 			viper.GetString("secret_name"),
-			viper.GetString("inline_scanning_extra_params"),
+			viper.GetString("cli_scanning_extra_params"),
 			viper.GetBool("verify_ssl"),
 			log.StandardLogger())
 	}
 
-	log.Info("Using backend-scanning adapter")
-	return scanner.NewBackendAdapter(client)
+	log.Fatal("Please specify the cli-scanner (--cli_scanning) command line parameter, backend scanning no longer supported")
+	return nil
 }
diff --git a/docs/images/secure_as_default_harbor_ui.png b/docs/images/secure_as_default_harbor_ui.png
diff --git a/docs/install.md b/docs/install.md
@@ -7,7 +7,8 @@ This guide explains how to install Harbor Scanner Adapter for Sysdig Secure.
 * Kubernetes >= 1.14
 * Harbor >= 1.10
 * Helm >= 3
-* A Sysdig Secure API Token
+* A valid Sysdig Secure API Token
+* A valid Sysdig URL
 
 ### Obtaining the Sysdig Secure API Token
 
@@ -16,20 +17,122 @@ settings. Is just below the Get Started sidebar item.
 
 ![Getting Secure API Token](images/getting_secure_api_token.png)
 
+### Obtaining the Sysdig Secure API Token
+Your URL is listed in the address bar of your browser.  If you login to `https://secure.sysdig.com` then that is your URL.
+If you login to `https://app.au1.sysdig.com` for the AP region, then this is the URL you use.
+
+### Example values.yaml configuration.
+
+```yaml
+# Default values for harbor-scanner-sysdig-secure.
+# This is a YAML-formatted file.
+# Declare variables to be passed into your templates.
+
+replicaCount: 1
+
+image:
+  repository: miles3719/harbor-scanner-sysdig-secure
+  pullPolicy: IfNotPresent
+
+imagePullSecrets: []
+nameOverride: ""
+fullnameOverride: ""
+
+podAnnotations: {}
+
+serviceAccount:
+  # Specifies whether a service account should be created
+  create: true
+  # Annotations to add to the service account
+  annotations: {}
+  # The name of the service account to use.
+  # If not set and create is true, a name is generated using the fullname template
+  name:
+
+rbac:
+  create: true
+
+podSecurityContext: {}
+# fsGroup: 2000
+
+securityContext: {}
+  # capabilities:
+  #   drop:
+  #   - ALL
+  # readOnlyRootFilesystem: true
+  # runAsNonRoot: true
+# runAsUser: 1000
+
+service:
+  type: ClusterIP
+  port: 5000
+
+resources: {}
+  # We usually recommend not to specify default resources and to leave this as a conscious
+  # choice for the user. This also increases chances charts run on environments with little
+  # resources, such as Minikube. If you do want to specify resources, uncomment the following
+  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
+  # limits:
+  #   cpu: 100m
+  #   memory: 128Mi
+  # requests:
+  #   cpu: 100m
+#   memory: 128Mi
+
+nodeSelector: {}
+
+tolerations: []
+
+affinity: {}
+
+# Custom entrypoint for the harbor plugin
+customEntryPoint: []
+
+sysdig:
+  secure:
+
+    # **required**
+    # API Token to access Sysdig Secure.
+    # If neither this value nor `sysdig.secure.existingSecureAPITokenSecret` are configured, the
+    # user will be required to provide the deployment the `SECURE_API_TOKEN` environment variables.
+    apiToken: bs456348-45a6-4b5f-c57d-35572b981a3b
+
+    # Alternatively, specify the name of a Kubernetes secret containing an 'sysdig_secure_api_token' entry
+    existingSecureAPITokenSecret: ""
+
+    # Sysdig backend URL (SaaS Regions API endpoints are listed here: https://docs.sysdig.com/en/docs/administration/saas-regions-and-ip-ranges/)
+    url: https://app.au1.sysdig.com
+    verifySSL: true
+
+proxy:
+  httpProxy:
+  httpsProxy:
+  # Comma-separated list of domain extensions proxy should not be used for.
+  # Include in noProxy the internal IP of the kubeapi server,
+  # and you probably need to add your registry if it is inside the cluster
+  noProxy:
+
+CliScanning:
+  enabled: true
+
+asyncMode:
+  enabled: true
+```
+
 ## Deploying on Kubernetes using the Helm Chart
 
 The fastest way to deploy the scanner adapter is using the Helm Chart we
 provide. Be aware that you need to provide the Sysdig Secure API token when
 you type the `helm install` command.
 
 ```
-$ helm repo add sysdiglabs https://sysdiglabs.github.io/charts
-"sysdiglabs" has been added to your repositories
+$ helm repo add aaronm-sysdig https://aaronm-sysdig.github.io/charts
+"aaronm-sysdig" has been added to your repositories
 
 $ kubectl create namespace harbor-scanner-sysdig-secure
 namespace/harbor-scanner-sysdig-secure created
 
-$ helm -n harbor-scanner-sysdig-secure install harbor-scanner-sysdig-secure --set sysdig.secure.apiToken=XXX sysdiglabs/harbor-scanner-sysdig-secure
+$ helm -n harbor-scanner-sysdig-secure install harbor-scanner-sysdig-secure --set sysdig.secure.apiToken=XXX aaronm-sysdig/harbor-scanner-sysdig-secure
 NAME: harbor-scanner-sysdig-secure
 LAST DEPLOYED: Tue Jun  9 13:38:12 2020
 NAMESPACE: harbor-scanner-sysdig-secure
@@ -46,22 +149,7 @@ kubectl --namespace harbor-scanner-sysdig-secure port-forward $POD_NAME 8080:80
 And that's it. The new scanner adapter is deployed. Now is time to tell Harbor
 to use it, and you can find [how to configure Harbor to use Sysdig Secure Scanner Adapter](#configuring-harbor-to-use-sysdig-secure-scanner-adapter) a few lines below.
 
-### Using Backend Scanning instead of Inline Scanning
-
-This mode is not recommended and it is supported only for legacy purposes.
-
-You will need to disable inline scan by setting `inlineScanning.enabled: false` in the values.yaml:
-
-```yaml
-sysdig:
-  secure:
-    apiToken: XXX
-
-inlineScanning:
-  enabled: false
-```
-
-You already know [how to get the Sysdig Secure API Token](#obtaining-the-sysdig-secure-api-token).
+You already know [how to get the Sysdig Secure API Token](#obtaining-the-sysdig-secure-api-token) and the Secure URL
 
 ## Configuring Harbor to use Sysdig Secure Scanner Adapter
 

diff --git a/go.mod b/go.mod
@@ -1,17 +1,75 @@
 module github.com/sysdiglabs/harbor-scanner-sysdig-secure
 
-go 1.16
+go 1.22.0
+
+toolchain go1.22.1
 
 require (
-	github.com/golang/mock v1.4.3
-	github.com/gorilla/handlers v1.4.2
-	github.com/gorilla/mux v1.7.4
+	github.com/golang/mock v1.6.0
+	github.com/gorilla/handlers v1.5.2
+	github.com/gorilla/mux v1.8.1
 	github.com/onsi/ginkgo v1.16.5
-	github.com/onsi/gomega v1.10.1
-	github.com/sirupsen/logrus v1.5.0
+	github.com/onsi/gomega v1.33.1
+	github.com/sirupsen/logrus v1.9.3
 	github.com/spf13/pflag v1.0.5
-	github.com/spf13/viper v1.7.0
-	k8s.io/api v0.19.13
-	k8s.io/apimachinery v0.19.13
-	k8s.io/client-go v0.19.13
+	github.com/spf13/viper v1.18.2
+	k8s.io/api v0.30.0
+	k8s.io/apimachinery v0.30.0
+	k8s.io/client-go v0.30.0
+)
+
+require (
+	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
+	github.com/emicklei/go-restful/v3 v3.12.0 // indirect
+	github.com/evanphx/json-patch v5.9.0+incompatible // indirect
+	github.com/felixge/httpsnoop v1.0.4 // indirect
+	github.com/fsnotify/fsnotify v1.7.0 // indirect
+	github.com/go-logr/logr v1.4.1 // indirect
+	github.com/go-openapi/jsonpointer v0.21.0 // indirect
+	github.com/go-openapi/jsonreference v0.21.0 // indirect
+	github.com/go-openapi/swag v0.23.0 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.4 // indirect
+	github.com/google/gnostic-models v0.6.8 // indirect
+	github.com/google/go-cmp v0.6.0 // indirect
+	github.com/google/gofuzz v1.2.0 // indirect
+	github.com/google/uuid v1.6.0 // indirect
+	github.com/hashicorp/hcl v1.0.0 // indirect
+	github.com/josharian/intern v1.0.0 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
+	github.com/magiconair/properties v1.8.7 // indirect
+	github.com/mailru/easyjson v0.7.7 // indirect
+	github.com/mitchellh/mapstructure v1.5.0 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
+	github.com/nxadm/tail v1.4.11 // indirect
+	github.com/pelletier/go-toml/v2 v2.2.2 // indirect
+	github.com/pkg/errors v0.9.1 // indirect
+	github.com/sagikazarmark/locafero v0.4.0 // indirect
+	github.com/sagikazarmark/slog-shim v0.1.0 // indirect
+	github.com/sourcegraph/conc v0.3.0 // indirect
+	github.com/spf13/afero v1.11.0 // indirect
+	github.com/spf13/cast v1.6.0 // indirect
+	github.com/subosito/gotenv v1.6.0 // indirect
+	go.uber.org/multierr v1.11.0 // indirect
+	golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 // indirect
+	golang.org/x/net v0.25.0 // indirect
+	golang.org/x/oauth2 v0.20.0 // indirect
+	golang.org/x/sys v0.20.0 // indirect
+	golang.org/x/term v0.20.0 // indirect
+	golang.org/x/text v0.15.0 // indirect
+	golang.org/x/time v0.5.0 // indirect
+	google.golang.org/protobuf v1.34.1 // indirect
+	gopkg.in/inf.v0 v0.9.1 // indirect
+	gopkg.in/ini.v1 v1.67.0 // indirect
+	gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.1 // indirect
+	k8s.io/klog/v2 v2.120.1 // indirect
+	k8s.io/kube-openapi v0.0.0-20240430033511-f0e62f92d13f // indirect
+	k8s.io/utils v0.0.0-20240502163921-fe8a2dddb1d0 // indirect
+	sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
+	sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
+	sigs.k8s.io/yaml v1.4.0 // indirect
 )