Skip to content

Commit

Permalink
feat(scanner): Change logic of creating entities (#309)
Browse files Browse the repository at this point in the history
* First commit

* Wip

* Revert f04feab

* Wip

* Wip

* Wip

* Wip

* Wip

* Wip

* Change naming

* Add concurrency

* Add READMEs

* Automatic application of license header

* Fix #309 (comment)

* Fix #discussion_r1808503568

* Fix #309 (comment)

* Fix #309 (comment)

* Automatic application of license header

* Fix #309 (comment)

* Refactoring

* Update dependencies

* Fix logic for ExtractImageInfo

* Add unit tests for ExtractImageInfo

* Automatic application of license header

* Clean-up

---------

Co-authored-by: License Bot <[email protected]>
Co-authored-by: Michael Reimsbach <[email protected]>
  • Loading branch information
3 people authored Oct 22, 2024
1 parent f423cf9 commit d26a8fb
Show file tree
Hide file tree
Showing 19 changed files with 959 additions and 219 deletions.
62 changes: 62 additions & 0 deletions scanner/k8s-assets/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Kubernetes Assets Scanner

The Kubernetes Assets Scanner is a tool designed to scan and collect information about services, pods, and containers running in a Kubernetes cluster. It processes the collected data and reports findings to a GraphQL API (presumably Heureka).

## Prerequisites

- Go 1.15 or later
- Access to a Kubernetes cluster
- Heureka system for reporting findings

## Installation

1. Clone the repository:
```
git clone https://github.com/cloudoperators/heureka.git
cd scanners/k8s-assets
```

2. Install dependencies:
```
go mod tidy
```

## Configuration

The scanner is configured using environment variables. Set the following variables before running the scanner:

- `HEUREKA_LOG_LEVEL`: Set the log level (default: "debug")
- `HEUREKA_KUBE_CONFIG_PATH`: Path to kubeconfig file (default: "~/.kube/config")
- `HEUREKA_KUBE_CONFIG_CONTEXT`: Kubernetes context to use
- `HEUREKA_KUBE_CONFIG_TYPE`: Type of Kubernetes config (default: "oidc")
- `HEUREKA_SUPPORT_GROUP_LABEL`: Label for support group (default: "ccloud/support-group")
- `HEUREKA_SERVICE_NAME_LABEL`: Label for service name (default: "ccloud/service")
- `HEUREKA_SCANNER_TIMEOUT`: Timeout for the scanner (default: "30m")
- `HEUREKA_HEUREKA_URL`: URL of the Heureka system for reporting findings
- `HEUREKA_CLUSTER_NAME`: Name of the cluster being scanned
- `HEUREKA_CLUSTER_REGION`: Region of the cluster being scanned

Example:

```bash
export HEUREKA_LOG_LEVEL=debug
export HEUREKA_KUBE_CONFIG_PATH=~/.kube/config
export HEUREKA_KUBE_CONFIG_CONTEXT=my-cluster-context
export HEUREKA_KUBE_CONFIG_TYPE=oidc
export HEUREKA_SUPPORT_GROUP_LABEL=ccloud/support-group
export HEUREKA_SERVICE_NAME_LABEL=ccloud/service
export HEUREKA_SCANNER_TIMEOUT=30m
export HEUREKA_HEUREKA_URL=https://heureka.example.com
export HEUREKA_CLUSTER_NAME=my-cluster
export HEUREKA_CLUSTER_REGION=us-west-1
```

## Usage

To run the Kubernetes Assets Scanner:

```bash
go run main.go
```

The scanner will start processing namespaces, services, pods, and containers, and report findings to the configured Heureka system.
96 changes: 78 additions & 18 deletions scanner/k8s-assets/client/generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions scanner/k8s-assets/client/query/component_create.graphql
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# SPDX-FileCopyrightText: 2024 SAP SE or an SAP affiliate company and Greenhouse contributors
# SPDX-License-Identifier: Apache-2.0

mutation CreateComponent($input: ComponentInput!) {
# @genqlient(typename: "Component")
createComponent (
input: $input
) {
id
name
type
}
}
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# SPDX-FileCopyrightText: 2024 SAP SE or an SAP affiliate company and Greenhouse contributors
# SPDX-License-Identifier: Apache-2.0

mutation CreateComponentVersion ($input: ComponentVersionInput!) {
mutation CreateComponentVersion($input: ComponentVersionInput!) {
# @genqlient(typename: "ComponentVersion")
createComponentVersion (
input: $input
) {
id
version
componentId
}
}
}
5 changes: 5 additions & 0 deletions scanner/k8s-assets/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,14 @@ require (
)

require (
github.com/agnivade/levenshtein v1.1.1 // indirect
github.com/alexflint/go-arg v1.4.2 // indirect
github.com/alexflint/go-scalar v1.0.0 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/google/pprof v0.0.0-20240525223248-4bfdf5a9a2af // indirect
github.com/vektah/gqlparser/v2 v2.5.15 // indirect
golang.org/x/mod v0.17.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
)

Expand Down
13 changes: 13 additions & 0 deletions scanner/k8s-assets/go.sum
Original file line number Diff line number Diff line change
@@ -1,12 +1,20 @@
github.com/Khan/genqlient v0.7.0 h1:GZ1meyRnzcDTK48EjqB8t3bcfYvHArCUUvgOwpz1D4w=
github.com/Khan/genqlient v0.7.0/go.mod h1:HNyy3wZvuYwmW3Y7mkoQLZsa/R5n5yIRajS1kPBvSFM=
github.com/agnivade/levenshtein v1.1.1 h1:QY8M92nrzkmr798gCo3kmMyqXFzdQVpxLlGPRBij0P8=
github.com/agnivade/levenshtein v1.1.1/go.mod h1:veldBMzWxcCG2ZvUTKD2kJNRdCk5hVbJomOvKkmgYbo=
github.com/alexflint/go-arg v1.4.2 h1:lDWZAXxpAnZUq4qwb86p/3rIJJ2Li81EoMbTMujhVa0=
github.com/alexflint/go-arg v1.4.2/go.mod h1:9iRbDxne7LcR/GSvEr7ma++GLpdIU1zrghf2y2768kM=
github.com/alexflint/go-scalar v1.0.0 h1:NGupf1XV/Xb04wXskDFzS0KWOLH632W/EO4fAFi+A70=
github.com/alexflint/go-scalar v1.0.0/go.mod h1:GpHzbCOZXEKMEcygYQ5n/aa4Aq84zbxjy3MxYW0gjYw=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883 h1:bvNMNQO63//z+xNgfBlViaCIJKLlCJ6/fmUseuG0wVQ=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/arbovm/levenshtein v0.0.0-20160628152529-48b4e1c0c4d0/go.mod h1:t2tdKJDJF9BV14lnkjHmOQgcvEKgtqs5a1N3LNdJhGE=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgryski/trifles v0.0.0-20200323201526-dd97f9abfb48/go.mod h1:if7Fbed8SFyPtHLHbg49SI7NAdJiC5WIA09pe59rfAA=
github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g=
github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E=
Expand Down Expand Up @@ -82,6 +90,7 @@ github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
Expand All @@ -100,6 +109,8 @@ golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8U
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
Expand All @@ -111,6 +122,8 @@ golang.org/x/oauth2 v0.21.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbht
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
Expand Down
Loading

0 comments on commit d26a8fb

Please sign in to comment.