Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(client): Allow agent to send cluster description #508

Merged
merged 1 commit into from
Feb 5, 2024
Merged

feat(client): Allow agent to send cluster description #508

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ require (
github.com/juju/errors v1.0.0
github.com/kylelemons/godebug v1.1.0
github.com/maxatome/go-testdeep v1.14.0
github.com/microcosm-cc/bluemonday v1.0.26
github.com/pkg/errors v0.9.1
github.com/pmylund/go-cache v2.1.0+incompatible
github.com/prometheus/client_golang v1.18.0
Expand All @@ -27,7 +28,9 @@ require (
)

require (
github.com/aymerick/douceur v0.2.0 // indirect
github.com/google/gnostic-models v0.6.9-0.20230804172637-c7be7c783f49 // indirect
github.com/gorilla/css v1.0.0 // indirect
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
golang.org/x/net v0.17.0 // indirect
)
Expand Down
14 changes: 14 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
github.com/Jeffail/gabs/v2 v2.7.0 h1:Y2edYaTcE8ZpRsR2AtmPu5xQdFDIthFG0jYhu5PY8kg=
github.com/Jeffail/gabs/v2 v2.7.0/go.mod h1:dp5ocw1FvBBQYssgHsG7I1WYsiLRtkUaB1FEtSwvNUw=
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
github.com/aymerick/douceur v0.2.0/go.mod h1:wlT5vV2O3h55X9m7iVYN0TBM0NH/MmbLnd30/FjWUq4=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cenkalti/backoff v2.2.1+incompatible h1:tNowT99t7UNflLxfYYSlKYsBpXdEet03Pg2g16Swow4=
Expand Down Expand Up @@ -31,6 +33,7 @@ github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+
github.com/go-openapi/swag v0.22.4 h1:QLMzNJnMGPRNDCbySlcj1x01tzU8/9LTTL9hZZZogBU=
github.com/go-openapi/swag v0.22.4/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang-jwt/jwt/v4 v4.5.0 h1:7cYmW1XlMY7h7ii7UhUyChSgS5wUJEnm9uZVTGqOWzg=
Expand All @@ -49,8 +52,11 @@ github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/
github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 h1:K6RDEckDVWvDI9JAJYCmNdQXq6neHJOYx3V6jnqNEec=
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/css v1.0.0 h1:BQqNyPTi50JCFMTw/b67hByjMVXZRwGha6wxVGkeihY=
github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
Expand All @@ -70,6 +76,7 @@ github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
Expand All @@ -87,6 +94,8 @@ github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 h1:jWpvCLoY8Z/e3VKvls
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0/go.mod h1:QUyp042oQthUoa9bqDv0ER0wrtXnBruoNd7aNjkbP+k=
github.com/maxatome/go-testdeep v1.14.0 h1:rRlLv1+kI8eOI3OaBXZwb3O7xY3exRzdW5QyX48g9wI=
github.com/maxatome/go-testdeep v1.14.0/go.mod h1:lPZc/HAcJMP92l7yI6TRz1aZN5URwUBUAfUNvrclaNM=
github.com/microcosm-cc/bluemonday v1.0.26 h1:xbqSvqzQMeEHCqMi64VAs4d8uy6Mequs3rQ0k/Khz58=
github.com/microcosm-cc/bluemonday v1.0.26/go.mod h1:JyzOCs9gkyQyjs+6h10UEVSe02CGwkhd72Xdqh78TWs=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
Expand All @@ -95,7 +104,9 @@ github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjY
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/onsi/ginkgo/v2 v2.9.4 h1:xR7vG4IXt5RWx6FfIjyAtsoMAtnc3C/rFXBBd2AjZwE=
github.com/onsi/ginkgo/v2 v2.9.4/go.mod h1:gCQYp2Q+kSoIj7ykSVb9nskRSsR6PUj4AiLywzIhbKM=
github.com/onsi/gomega v1.27.6 h1:ENqfyGeS5AX/rlXDd/ETokDz93u0YufY1Pgxuy/PvWE=
github.com/onsi/gomega v1.27.6/go.mod h1:PIQNjfQwkP3aQAH7lf7j87O/5FiNr+ZR8+ipb+qQlhg=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
Expand All @@ -111,6 +122,7 @@ github.com/prometheus/common v0.45.0/go.mod h1:YJmSTw9BoKxJplESWWxlbyttQR4uaEcGy
github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo=
github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
Expand All @@ -124,6 +136,7 @@ github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
Expand Down Expand Up @@ -177,6 +190,7 @@ golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roY
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.8.0 h1:vSDcovVPld282ceKgDimkRSC8kpaH1dgyc9UMzlt84Y=
golang.org/x/tools v0.8.0/go.mod h1:JxBZ99ISMI5ViVkT1tr6tdNmXeTrcpVSD3vZ1RsRdN4=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
Expand Down
5 changes: 3 additions & 2 deletions pkg/agent/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ type Config struct {
// OrganizationID within Preflight that will receive the data.
OrganizationID string `yaml:"organization_id"`
// ClusterID is the cluster that the agent is scanning.
ClusterID string `yaml:"cluster_id"`
DataGatherers []DataGatherer `yaml:"data-gatherers"`
ClusterID string `yaml:"cluster_id"`
ClusterDescription string `yaml:"cluster_description"`
DataGatherers []DataGatherer `yaml:"data-gatherers"`
// InputPath replaces DataGatherers with input data file
InputPath string `yaml:"input-path"`
// OutputPath replaces Server with output data file
Expand Down
5 changes: 4 additions & 1 deletion pkg/agent/run.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -411,7 +411,10 @@ func postData(config Config, preflightClient client.Client, readings []*api.Data

if VenafiCloudMode {
// orgID and clusterID are not required for Venafi Cloud auth
err := preflightClient.PostDataReadings("", "", readings)
err := preflightClient.PostDataReadingsWithOptions(readings, client.Options{
ClusterName: config.ClusterID,
ClusterDescription: config.ClusterDescription,
})
if err != nil {
return fmt.Errorf("post to server failed: %+v", err)
}
Expand Down
10 changes: 10 additions & 0 deletions pkg/client/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,19 @@ import (
)

type (
// Options is the struct describing additional information pertinent to an agent that isn't a data reading
// These fields will then be uploaded together with data readings.
Options struct {
OrgID string
ClusterID string
ClusterName string
Comment on lines +17 to +18
Copy link
Member

@maelvls maelvls Jul 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between ClusterName and ClusterID? It seems like ClusterName is set to ClusterID in run.go:

err := preflightClient.PostDataReadingsWithOptions(readings, client.Options{
	ClusterName:        config.ClusterID,
	ClusterDescription: config.ClusterDescription,
})

and ClusterID isn't set? I would have expected something like:

err := preflightClient.PostDataReadingsWithOptions(readings, client.Options{
	ClusterName:        config.ClusterID,
	ClusterID:          config.ClusterID,
	ClusterDescription: config.ClusterDescription,
})

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking up earlier in the file run.go:

// orgID and clusterID are not required for Venafi Cloud auth

Sounds like we could comment the Options struct like this:

	Options struct {
		OrgID              string // Only used with platform.jetstack.io
		ClusterID          string // Only used with platform.jetstack.io
		ClusterName        string // Only used with Venafi Cloud
		ClusterDescription string // Only used with Venafi Cloud
	}

ClusterDescription string
}

// The Client interface describes types that perform requests against the Jetstack Secure backend.
Client interface {
PostDataReadings(orgID, clusterID string, readings []*api.DataReading) error
PostDataReadingsWithOptions(readings []*api.DataReading, options Options) error
Post(path string, body io.Reader) (*http.Response, error)
}

Expand Down
6 changes: 6 additions & 0 deletions pkg/client/client_api_token.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ func NewAPITokenClient(agentMetadata *api.AgentMetadata, apiToken, baseURL strin
}, nil
}

// PostDataReadingsWithOptions uploads the slice of api.DataReading to the Jetstack Secure backend to be processed for later
// viewing in the user-interface.
func (c *APITokenClient) PostDataReadingsWithOptions(readings []*api.DataReading, opts Options) error {
return c.PostDataReadings(opts.OrgID, opts.ClusterID, readings)
}

// PostDataReadings uploads the slice of api.DataReading to the Jetstack Secure backend to be processed for later
// viewing in the user-interface.
func (c *APITokenClient) PostDataReadings(orgID, clusterID string, readings []*api.DataReading) error {
Expand Down
4 changes: 4 additions & 0 deletions pkg/client/client_oauth.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,10 @@ func NewOAuthClient(agentMetadata *api.AgentMetadata, credentials *OAuthCredenti
}, nil
}

func (c *OAuthClient) PostDataReadingsWithOptions(readings []*api.DataReading, opts Options) error {
return c.PostDataReadings(opts.OrgID, opts.ClusterID, readings)
}

// PostDataReadings uploads the slice of api.DataReading to the Jetstack Secure backend to be processed for later
// viewing in the user-interface.
func (c *OAuthClient) PostDataReadings(orgID, clusterID string, readings []*api.DataReading) error {
Expand Down
4 changes: 4 additions & 0 deletions pkg/client/client_unauthenticated.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ func NewUnauthenticatedClient(agentMetadata *api.AgentMetadata, baseURL string)
}, nil
}

func (c *UnauthenticatedClient) PostDataReadingsWithOptions(readings []*api.DataReading, opts Options) error {
return c.PostDataReadings(opts.OrgID, opts.ClusterID, readings)
}

// PostDataReadings uploads the slice of api.DataReading to the Jetstack Secure backend to be processed for later
// viewing in the user-interface.
func (c *UnauthenticatedClient) PostDataReadings(orgID, clusterID string, readings []*api.DataReading) error {
Expand Down
53 changes: 53 additions & 0 deletions pkg/client/client_venafi_cloud.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"crypto/ed25519"
"crypto/rsa"
"crypto/x509"
"encoding/base64"
"encoding/json"
"encoding/pem"
"fmt"
Expand All @@ -25,6 +26,7 @@ import (
"github.com/google/uuid"
"github.com/hashicorp/go-multierror"
"github.com/jetstack/preflight/api"
"github.com/microcosm-cc/bluemonday"
)

type (
Expand Down Expand Up @@ -152,6 +154,57 @@ func (c *VenafiSvcAccountCredentials) IsClientSet() bool {
return c.ClientID != "" && c.PrivateKeyFile != ""
}

// PostDataReadingsWithOptions uploads the slice of api.DataReading to the Venafi Cloud backend to be processed.
// The Options are then passed as URL params in the request
func (c *VenafiCloudClient) PostDataReadingsWithOptions(readings []*api.DataReading, opts Options) error {
payload := api.DataReadingsPost{
AgentMetadata: c.agentMetadata,
DataGatherTime: time.Now().UTC(),
DataReadings: readings,
}
data, err := json.Marshal(payload)
if err != nil {
return err
}

if !strings.HasSuffix(c.uploadPath, "/") {
c.uploadPath = fmt.Sprintf("%s/", c.uploadPath)
}

venafiCloudUploadURL, err := url.Parse(filepath.Join(c.uploadPath, c.uploaderID))
if err != nil {
return err
}

// validate options and send them as URL params
query := venafiCloudUploadURL.Query()
stripHTML := bluemonday.StrictPolicy()
if opts.ClusterName != "" {
query.Add("name", stripHTML.Sanitize(opts.ClusterName))
}
Comment on lines +182 to +184
Copy link
Member

@maelvls maelvls Jul 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Passing by) Just for future reference, I was wondering this: why do we pass the query param name set to the config's cluster_id? The cluster_id is already present in the request body of /v1/tlspk/upload/clusterdata/:uploaderId, more specifically under agent_metadata. For example:

POST https://api.venafi.cloud/v1/tlspk/upload/clusterdata/no?name=mvalais+Kind+Cluster HTTP/2.0

content-type: application/json
authorization: Bearer ...
content-length: 536
accept-encoding: gzip
user-agent: Go-http-client/2.0
{
  "agent_metadata": {
    "version": "development",
    "cluster_id": "mvalais Kind Cluster"
  },
  "data_gather_time": "2024-07-26T13:03:39.831129Z",
  "data_readings": [
    {
      "cluster_id": "mvalais Kind Cluster",
      "data-gatherer": "k8s-discovery",
      "timestamp": "2024-07-26T15:03:09+02:00",
      "data": {
        "server_version": {
          "major": "1",
          "minor": "29",
          "gitVersion": "v1.29.2",
          "gitCommit": "4b8e819355d791d96b7e9d9efe4cbafae2311c88",
          "gitTreeState": "clean",
          "buildDate": "2024-02-14T22:25:42Z",
          "goVersion": "go1.21.7",
          "compiler": "gc",
          "platform": "linux/arm64"
        }
      },
      "schema_version": "v2.0.0"
    }
  ]
}

When I don't pass the query param name=mvalais+Kind+Cluster, I get the following error:

[{"message":"could not find a Kubernetes cluster linked to the client in the Venafi Control Plane"}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The payload body could be quite large, several MB depending on the cluster where the agent is installed, in the backend entrypoint we wanted to be able to get this information without having to parse the entire upload before the actual processing pipeline kicks in. (Note: we could stream the upload JSON but we thought query params would work well instead).

The error can be improved, but essentially you have 2 options to onboard a cluster:

  • venctl installed which creates a service account and cluster_connection (relation of cluster and service account) in VCP.
  • Helm installation which requires the backend entrypoint to create a cluster_connection in VCP for your cluster and service account.

The error is saying that the cluster and service account are not linked in VCP.

In the venafi docs we tell users following the helm installation that they are required to set the cluster name, which will also be present in the UI. We could introduce an auto-onboard field to the config but since the cluster name field is required for the helm installation we thought we could use it rather than introducing another field.

If the cluster name is set in the config it will be used in the query param. The user wouldn't interact with the query params.

Copy link
Member

@maelvls maelvls Jul 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I didn't think of the body lenght of the request, and since the server doesn't parse JSON as a stream, it makes sense to use a query parameter. Super useful insight, thanks!

Comment on lines +181 to +184
Copy link
Member

@maelvls maelvls Jul 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Passing by) Why was stripHTML.Sanitize needed here? Isn't overkill in this context? query.Encode already URL-encodes values, and this value isn't meant to be interpreted by any HTML parsers

I'd expect this type of sanitization to happen on the server side, since users may use the API directly (without us knowing) and they may inject things in the cluster name, such as HTML tags.

if opts.ClusterDescription != "" {
query.Add("description", base64.RawURLEncoding.EncodeToString([]byte(stripHTML.Sanitize(opts.ClusterDescription))))
}
Comment on lines +185 to +187
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why base 64 encode this? Query parameters are already URL-encoded by query.Encode, which should be enough to pass that data to the server.

Same question as above regarding stripHTML.Sanitize too

venafiCloudUploadURL.RawQuery = query.Encode()

res, err := c.Post(venafiCloudUploadURL.String(), bytes.NewBuffer(data))
if err != nil {
return err
}
defer res.Body.Close()

if code := res.StatusCode; code < 200 || code >= 300 {
errorContent := ""
body, err := io.ReadAll(res.Body)
if err == nil {
errorContent = string(body)
}
return fmt.Errorf("received response with status code %d. Body: %s", code, errorContent)
}

return nil
}

// PostDataReadings uploads the slice of api.DataReading to the Venafi Cloud backend to be processed for later
// viewing in the user-interface.
func (c *VenafiCloudClient) PostDataReadings(_ string, _ string, readings []*api.DataReading) error {
Expand Down
Loading