Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Auditbeat/FIM/fsnotify]: remove time window where a child file operation of a directory can be lost #39133

Conversation

pkoutsovasilis
Copy link
Contributor

@pkoutsovasilis pkoutsovasilis commented Apr 22, 2024

Proposed commit message

This PR fixes lost events for file integrity module of Auditbeat. Specifically as captured here there is tiny time window where if a child file get's created right after the internal dir contents snapshot of filepath.Walk and before we add the path to the fsnotify watcher, this file is now off the radar.

Mutliple of this tiny windows have been present during some buildkite CI runs (#39110 (comment), #39109 (comment), #39108 (comment))

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

N/A

How to test this PR locally

cd auditbeat && for i in {1..15}; do mage build unitTest; done

Related issues

Use cases

N/A

Screenshots

N/A

Logs

N/A

@pkoutsovasilis pkoutsovasilis added backport-7.17 Automated backport to the 7.17 branch with mergify bugfix Team:Security-Linux Platform Linux Platform Team in Security Solution backport-v8.13.0 Automated backport with mergify labels Apr 22, 2024
@pkoutsovasilis pkoutsovasilis self-assigned this Apr 22, 2024
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Apr 22, 2024
@pkoutsovasilis pkoutsovasilis force-pushed the pkoutsovasilis/fix_fsnotify_recursive branch from c7759a6 to b56f1d6 Compare April 22, 2024 15:24
@pkoutsovasilis pkoutsovasilis force-pushed the pkoutsovasilis/fix_fsnotify_recursive branch from b56f1d6 to ed523b2 Compare April 22, 2024 15:25
@elastic elastic deleted a comment from mergify bot Apr 22, 2024
@pkoutsovasilis pkoutsovasilis marked this pull request as ready for review April 22, 2024 15:28
@pkoutsovasilis pkoutsovasilis requested a review from a team as a code owner April 22, 2024 15:28
@elasticmachine
Copy link
Collaborator

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 22, 2024

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 89 min 14 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@pkoutsovasilis pkoutsovasilis added the backport-v8.14.0 Automated backport with mergify label Apr 22, 2024
@pkoutsovasilis
Copy link
Contributor Author

run docs-build

Copy link
Contributor

@Tacklebox Tacklebox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -202,7 +207,7 @@ func TestRecursiveSubdirPermissions(t *testing.T) {
for {
// No event is received
ev, err := readTimeout(t, watcher)
if err == errReadTimeout {
if errors.Is(err, errReadTimeout) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to spend more time learning go-isms.

@pkoutsovasilis pkoutsovasilis merged commit 074f221 into elastic:main Apr 26, 2024
39 checks passed
mergify bot pushed a commit that referenced this pull request Apr 26, 2024
…tion of a directory can be lost (#39133)

* fix(auditbeat/fim/fsnotify): remove time window where a child file operation in of a directory can be lost

* fix(auditbeat/fim/fsnotify): remove redundant error wrapping

(cherry picked from commit 074f221)

# Conflicts:
#	auditbeat/module/file_integrity/monitor/recursive.go
mergify bot pushed a commit that referenced this pull request Apr 26, 2024
…tion of a directory can be lost (#39133)

* fix(auditbeat/fim/fsnotify): remove time window where a child file operation in of a directory can be lost

* fix(auditbeat/fim/fsnotify): remove redundant error wrapping

(cherry picked from commit 074f221)
mergify bot pushed a commit that referenced this pull request Apr 26, 2024
…tion of a directory can be lost (#39133)

* fix(auditbeat/fim/fsnotify): remove time window where a child file operation in of a directory can be lost

* fix(auditbeat/fim/fsnotify): remove redundant error wrapping

(cherry picked from commit 074f221)
pkoutsovasilis added a commit that referenced this pull request Apr 30, 2024
…where a child file operation of a directory can be lost (#39231)

* [Auditbeat/FIM/fsnotify]: remove time window where a child file operation of a directory can be lost (#39133)

* fix(auditbeat/fim/fsnotify): remove time window where a child file operation in of a directory can be lost

* fix(auditbeat/fim/fsnotify): remove redundant error wrapping

(cherry picked from commit 074f221)

# Conflicts:
#	auditbeat/module/file_integrity/monitor/recursive.go

* fix: resolve conflicts

---------

Co-authored-by: Panos Koutsovasilis <[email protected]>
pkoutsovasilis added a commit that referenced this pull request Apr 30, 2024
…tion of a directory can be lost (#39133) (#39233)

* fix(auditbeat/fim/fsnotify): remove time window where a child file operation in of a directory can be lost

* fix(auditbeat/fim/fsnotify): remove redundant error wrapping

(cherry picked from commit 074f221)

Co-authored-by: Panos Koutsovasilis <[email protected]>
pkoutsovasilis added a commit that referenced this pull request Apr 30, 2024
…tion of a directory can be lost (#39133) (#39232)

* fix(auditbeat/fim/fsnotify): remove time window where a child file operation in of a directory can be lost

* fix(auditbeat/fim/fsnotify): remove redundant error wrapping

(cherry picked from commit 074f221)

Co-authored-by: Panos Koutsovasilis <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.13.0 Automated backport with mergify backport-v8.14.0 Automated backport with mergify bugfix Team:Security-Linux Platform Linux Platform Team in Security Solution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Auditbeat/fim/fsnotify]: tiny time window that loses file events
5 participants