Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: stats in webhook, audit & gator #2686

Merged
merged 15 commits into from
May 9, 2023

Conversation

acpana
Copy link
Contributor

@acpana acpana commented Apr 7, 2023

Integrates stats under flags for admission, audit & gator test. This should be viewed as an iterative process and we can revisit the emission mechanism (ie logging) or think about aggregation or prometheus metrics, etc. in follow ups.

Docs will come as a follow up once code lands.

cmd/gator/test/test.go Outdated Show resolved Hide resolved
cmd/gator/test/test.go Outdated Show resolved Hide resolved
cmd/gator/test/test.go Outdated Show resolved Hide resolved
pkg/audit/manager.go Outdated Show resolved Hide resolved
pkg/audit/manager.go Outdated Show resolved Hide resolved
pkg/webhook/policy.go Show resolved Hide resolved
pkg/webhook/policy.go Show resolved Hide resolved
@acpana acpana mentioned this pull request Apr 13, 2023
1 task
@acpana acpana force-pushed the acpana/gator-stats branch from 1ce1c9f to e85f1b1 Compare April 13, 2023 22:27
@acpana acpana changed the title feat: stats in gator feat: stats in webhook, audit & gator Apr 13, 2023
@acpana acpana requested a review from maxsmythe April 13, 2023 23:27
@acpana acpana force-pushed the acpana/gator-stats branch from e85f1b1 to 14c792f Compare April 14, 2023 22:22
@acpana acpana marked this pull request as ready for review April 14, 2023 22:37
@codecov-commenter
Copy link

codecov-commenter commented Apr 19, 2023

Codecov Report

Attention: Patch coverage is 46.47887% with 76 lines in your changes missing coverage. Please review.

Project coverage is 52.80%. Comparing base (74dd0d4) to head (74ff964).
Report is 354 commits behind head on master.

Files with missing lines Patch % Lines
cmd/gator/test/test.go 25.86% 37 Missing and 6 partials ⚠️
pkg/audit/manager.go 0.00% 16 Missing ⚠️
pkg/webhook/policy.go 0.00% 15 Missing and 1 partial ⚠️
pkg/gator/test/test.go 92.85% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2686      +/-   ##
==========================================
+ Coverage   52.76%   52.80%   +0.04%     
==========================================
  Files         123      125       +2     
  Lines       10941    11053     +112     
==========================================
+ Hits         5773     5837      +64     
- Misses       4711     4759      +48     
  Partials      457      457              
Flag Coverage Δ
unittests 52.80% <46.47%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@acpana
Copy link
Contributor Author

acpana commented Apr 20, 2023

@maxsmythe PTAL 🙏🏼

cmd/gator/commons/commons.go Outdated Show resolved Hide resolved
cmd/gator/commons/commons.go Outdated Show resolved Hide resolved
cmd/gator/test/test.go Outdated Show resolved Hide resolved
cmd/gator/test/test.go Outdated Show resolved Hide resolved
cmd/gator/test/test.go Outdated Show resolved Hide resolved
pkg/gator/test/test.go Outdated Show resolved Hide resolved
pkg/gator/test/test.go Outdated Show resolved Hide resolved
pkg/logging/logging.go Outdated Show resolved Hide resolved
pkg/expansion/aggregate.go Outdated Show resolved Hide resolved
pkg/gator/test/test.go Show resolved Hide resolved
cmd/gator/test/test.go Outdated Show resolved Hide resolved
cmd/gator/test/test.go Outdated Show resolved Hide resolved
@davis-haba
Copy link
Contributor

davis-haba commented Apr 21, 2023

LGTM with nits (after Max's comments are addressed about the output file flag and skipping self-validation in g8r)

We should update the docs too, but this can be done as a follow up if you prefer.

Signed-off-by: Alex Pana <[email protected]>
@acpana acpana force-pushed the acpana/gator-stats branch from 92ab0e8 to f2f1026 Compare April 24, 2023 20:28
@acpana
Copy link
Contributor Author

acpana commented Apr 24, 2023

regarding

We should update the docs too, but this can be done as a follow up if you prefer.

I was planning on updating the docs separately.

@acpana acpana requested review from davis-haba and maxsmythe April 26, 2023 16:23
cmd/gator/test/test.go Outdated Show resolved Hide resolved
cmd/gator/test/test.go Outdated Show resolved Hide resolved
@acpana acpana force-pushed the acpana/gator-stats branch from 7e840bb to 8268335 Compare May 2, 2023 22:48
@ritazh ritazh added this to the v3.13.0 milestone May 3, 2023
pkg/logging/logging.go Outdated Show resolved Hide resolved
acpana added 3 commits May 4, 2023 00:18
Signed-off-by: Alex Pana <[email protected]>
Signed-off-by: Alex Pana <[email protected]>
@acpana acpana requested a review from maxsmythe May 4, 2023 01:43
Copy link
Contributor

@maxsmythe maxsmythe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@acpana acpana requested review from ritazh and sozercan May 4, 2023 01:48
@acpana
Copy link
Contributor Author

acpana commented May 8, 2023

@sozercan @ritazh PTAL when you have a chance 🙏🏼 💯

}

func (c *fakeCore) Sync() error {
return nil // TODO(acpana): revisit implementation for GC
Copy link
Member

@sozercan sozercan May 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

before we merge, can we open an issue tracking this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pkg/logging/logging.go Outdated Show resolved Hide resolved
@@ -62,6 +62,7 @@ var (
auditMatchKindOnly = flag.Bool("audit-match-kind-only", false, "only use kinds specified in all constraints for auditing cluster resources. if kind is not specified in any of the constraints, it will audit all resources (same as setting this flag to false)")
apiCacheDir = flag.String("api-cache-dir", defaultAPICacheDir, "The directory where audit from api server cache are stored, defaults to /tmp/audit")
emptyAuditResults []updateListEntry
logStatsAudit = flag.Bool("log-stats-audit", false, "(alpha) log stats metrics for the audit run")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a doc to describe usage? can be a follow-up

)

func init() {
Cmd.Flags().StringArrayVarP(&flagFilenames, flagNameFilename, "f", []string{}, "a file or directory containing Kubernetes resources. Can be specified multiple times.")
Cmd.Flags().StringVarP(&flagOutput, flagNameOutput, "o", "", fmt.Sprintf("Output format. One of: %s|%s.", stringJSON, stringYAML))
Cmd.Flags().BoolVarP(&flagIncludeTrace, "trace", "t", false, `include a trace for the underlying constraint framework evaluation`)
Cmd.Flags().BoolVarP(&flagIncludeTrace, "trace", "t", false, "include a trace for the underlying Constraint Framework evaluation.")
Copy link
Member

@sozercan sozercan May 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add a doc that describes usage? can be a follow up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes for stats feature I was going to do a follow up docs PR.

Copy link
Member

@sozercan sozercan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few minor comments, otherwise LGTM

pkg/logging/logging.go Outdated Show resolved Hide resolved
pkg/logging/logging_test.go Outdated Show resolved Hide resolved
pkg/logging/logging.go Outdated Show resolved Hide resolved
acpana and others added 2 commits May 9, 2023 12:10
Co-authored-by: Sertaç Özercan <[email protected]>
Signed-off-by: alex <[email protected]>
@maxsmythe maxsmythe merged commit a6bf666 into open-policy-agent:master May 9, 2023
anlandu pushed a commit to anlandu/gatekeeper that referenced this pull request May 19, 2023
Signed-off-by: Alex Pana <[email protected]>
Signed-off-by: alex <[email protected]>
Co-authored-by: Sertaç Özercan <[email protected]>
Signed-off-by: Anlan Du <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants