Skip to content

Commit

Permalink
[Auditbeat] fim(ebpf): enrich file events with process data (#38199)
Browse files Browse the repository at this point in the history 
* fim(ebpf): enrich file events with process data

* apply review suggestions

* apply review suggestions

* fix(fim/ebpf): move process fields to event root and insert them so keys do not contain dots

* fix(fim/ebpf): refactor HostID to utilise sync.OnceValue and expose boot time

* fix(fim/ebpf): refactor TicksPerSecond to utilise sync.OnceValue

* fix(fim/ebpf): remove empty slice allocation

* chore: go mod tidy

* fix: explicitly set go 1.21.8 in go.mod

* fix(fim/ebpf): nil slice of errors in TestNewEventFromEbpfEvent

* fix(fim/ebpf): remove re-declaration of already ecs included fields

* fix(fim/ebpf): utilise OnceValues to declutter the code

* fix(fim/ebpf): remove x-pack import from OSS package

* fix(fim/ebpf): propagate process fields changes to integration tests

* chore: go mod tidy

* ci: temporary solution to outdated docker compose python library

* ci: transition to a fixed tag for docker image instead of a rolling one

---------

Co-authored-by: Panos Koutsovasilis <[email protected]>
Co-authored-by: Pierre HILBERT <[email protected]>
  • Loading branch information
@mmat11 @pkoutsovasilis @pierrehilbert
3 people authored Apr 5, 2024
1 parent 80dab50 commit dbdaac3
Show file tree
Hide file tree
Showing 13 changed files with 431 additions and 107 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.next.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,9 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]

*Auditbeat*

- Add linux capabilities to processes in the system/process. {pull}37453[37453]
- Add opt-in eBPF backend for file_integrity module. {pull}37223[37223]
- Add process data to file events (Linux only, eBPF backend). {pull}38199[38199]

*Filebeat*

Expand Down
146 changes: 78 additions & 68 deletions NOTICE.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12287,11 +12287,11 @@ SOFTWARE.

--------------------------------------------------------------------------------
Dependency : github.com/elastic/ebpfevents
Version: v0.4.0
Version: v0.5.0
Licence type (autodetected): Apache-2.0
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/elastic/ebpfevents@v0.4.0/LICENSE.txt:
Contents of probable licence file $GOMODCACHE/github.com/elastic/ebpfevents@v0.5.0/LICENSE.txt:

The https://github.com/elastic/ebpfevents repository contains source code under
various licenses:
Expand Down Expand Up @@ -22921,6 +22921,45 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


--------------------------------------------------------------------------------
Dependency : github.com/tklauser/go-sysconf
Version: v0.3.10
Licence type (autodetected): BSD-3-Clause
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/tklauser/[email protected]/LICENSE:

BSD 3-Clause License

Copyright (c) 2018-2021, Tobias Klauser
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--------------------------------------------------------------------------------
Dependency : github.com/tsg/go-daemon
Version: v0.0.0-20200207173439-e704b93fd89b
Expand Down Expand Up @@ -36661,11 +36700,11 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

--------------------------------------------------------------------------------
Dependency : github.com/cilium/ebpf
Version: v0.12.3
Version: v0.13.2
Licence type (autodetected): MIT
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/cilium/ebpf@v0.12.3/LICENSE:
Contents of probable licence file $GOMODCACHE/github.com/cilium/ebpf@v0.13.2/LICENSE:

MIT License

Expand Down Expand Up @@ -38697,11 +38736,11 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--------------------------------------------------------------------------------
Dependency : github.com/frankban/quicktest
Version: v1.14.5
Version: v1.14.3
Licence type (autodetected): MIT
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/frankban/[email protected].5/LICENSE:
Contents of probable licence file $GOMODCACHE/github.com/frankban/[email protected].3/LICENSE:

MIT License

Expand Down Expand Up @@ -39304,6 +39343,37 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


--------------------------------------------------------------------------------
Dependency : github.com/go-quicktest/qt
Version: v1.101.0
Licence type (autodetected): MIT
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/go-quicktest/[email protected]/LICENSE:

MIT License

Copyright (c) 2017 Canonical Ltd.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


--------------------------------------------------------------------------------
Dependency : github.com/go-sourcemap/sourcemap
Version: v2.1.2+incompatible
Expand Down Expand Up @@ -49663,27 +49733,6 @@ OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--------------------------------------------------------------------------------
Dependency : github.com/pkg/diff
Version: v0.0.0-20210226163009-20ebb0f2a09e
Licence type (autodetected): BSD-3-Clause
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/pkg/[email protected]/LICENSE:

Copyright 2018 Joshua Bleecher Snyder

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--------------------------------------------------------------------------------
Dependency : github.com/pmezard/go-difflib
Version: v1.0.0
Expand Down Expand Up @@ -49967,11 +50016,11 @@ Contents of probable licence file $GOMODCACHE/github.com/prometheus/client_golan

--------------------------------------------------------------------------------
Dependency : github.com/rogpeppe/go-internal
Version: v1.9.0
Version: v1.11.0
Licence type (autodetected): BSD-3-Clause
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/rogpeppe/go-internal@v1.9.0/LICENSE:
Contents of probable licence file $GOMODCACHE/github.com/rogpeppe/go-internal@v1.11.0/LICENSE:

Copyright (c) 2018 The Go Authors. All rights reserved.

Expand Down Expand Up @@ -50873,45 +50922,6 @@ IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


--------------------------------------------------------------------------------
Dependency : github.com/tklauser/go-sysconf
Version: v0.3.10
Licence type (autodetected): BSD-3-Clause
--------------------------------------------------------------------------------

Contents of probable licence file $GOMODCACHE/github.com/tklauser/[email protected]/LICENSE:

BSD 3-Clause License

Copyright (c) 2018-2021, Tobias Klauser
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

* Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--------------------------------------------------------------------------------
Dependency : github.com/tklauser/numcpus
Version: v0.4.0
Expand Down
46 changes: 46 additions & 0 deletions auditbeat/module/file_integrity/event.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -134,13 +134,41 @@ type Event struct {
Action Action `json:"action"` // Action (like created, updated).
Hashes map[HashType]Digest `json:"hash,omitempty"` // File hashes.
ParserResults mapstr.M `json:"file,omitempty"` // Results from running file parsers.
Process *Process `json:"process,omitempty"` // Process data. Available only on Linux when using the eBPF backend.

// Metadata
rtt time.Duration // Time taken to collect the info.
errors []error // Errors that occurred while collecting the info.
hashFailed bool // Set when hashing the file failed.
}

// Process contain information about a process.
// These fields can help you correlate metrics information with a process id/name from a log message. The `process.pid` often stays in the metric itself and is copied to the global field for correlation.
type Process struct {
// Unique identifier for the process.
// The implementation of this is specified by the data source, but some examples of what could be used here are a process-generated UUID, Sysmon Process GUIDs, or a hash of some uniquely identifying components of a process.
// Constructing a globally unique identifier is a common practice to mitigate PID reuse as well as to identify a specific process over time, across multiple monitored hosts.
EntityID string `json:"entity_id,omitempty"`
// Process name. Sometimes called program name or similar.
Name string `json:"name,omitempty"`
// The effective user (euid).
User struct {
// Unique identifier of the user.
ID string `json:"id,omitempty"`
// Short name or login of the user.
Name string `json:"name,omitempty"`
} `json:"user,omitempty"`
// The effective group (egid).
Group struct {
// Unique identifier for the group on the system/platform.
ID string `json:"id,omitempty"`
// Name of the group.
Name string `json:"name,omitempty"`
} `json:"group,omitempty"`
// Process id.
PID uint32 `json:"pid,omitempty"`
}

// Metadata contains file metadata.
type Metadata struct {
Inode uint64 `json:"inode"`
Expand Down Expand Up @@ -354,6 +382,24 @@ func buildMetricbeatEvent(e *Event, existedBefore bool) mb.Event {
}
}

if e.Process != nil {
process := mapstr.M{
"pid": e.Process.PID,
"name": e.Process.Name,
"entity_id": e.Process.EntityID,
"user": mapstr.M{
"id": e.Process.User.ID,
"name": e.Process.User.Name,
},
"group": mapstr.M{
"id": e.Process.Group.ID,
"name": e.Process.Group.Name,
},
}

out.MetricSetFields.Put("process", process)
}

if len(e.Hashes) > 0 {
hashes := make(mapstr.M, len(e.Hashes))
for hashType, digest := range e.Hashes {
Expand Down
Loading

0 comments on commit dbdaac3

Please sign in to comment.