Incompatability between libbpf and libbpfgo #413

Hugo96870 · 2024-03-19T21:30:35Z

Hugo96870
Mar 19, 2024

Hey!

I installed cilium and the go library

Now I'm trying to run the program but I'm getting incompatibility issues.

ebpf program

#include "vmlinux.h"
#include <bpf/bpf_helpers.h>

// Define a map to hold the count of bytes written
struct {
    __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
    __type(key, u32);
    __type(value, long);
    __uint(max_entries, 64);
} write_count SEC(".maps");

SEC("kprobe/sys_write")
int bpf_prog1(const struct pt_regs *ctx)
{
    u32 key = 0;
    long bytes_written;

    bpf_probe_read(&bytes_written, sizeof(bytes_written), (void *)ctx + 24);

    // Update the map with the count of bytes written
    bpf_map_update_elem(&write_count, &key, &bytes_written, BPF_ANY);

    return 0;
}

char LICENSE[] SEC("license") = "GPL";

Go loader:

package main

import (
    "fmt"
    "os"
    "os/signal"
    "syscall"
    "github.com/aquasecurity/tracee/libbpfgo"
)

func main() {
    // Load the BPF Object
    bpfModule, err := libbpfgo.NewModuleFromFile("write_trace.bpf.o")
    if err != nil {
        fmt.Fprintf(os.Stderr, "could not load bpf module: %v\n", err)
        os.Exit(1)
    }
    defer bpfModule.Close()

    // Load and attach tracepoint
    err = bpfModule.BPFLoadObject()
    if err != nil {
        fmt.Fprintf(os.Stderr, "could not load bpf object: %v\n", err)
        os.Exit(1)
    }

	prog, err := bpfModule.GetProgram("bpf_prog1")
	if err != nil {
		fmt.Fprintf(os.Stderr, "could not get BPF program: %v\n", err)
		os.Exit(1)
	}
	
	_, err = prog.AttachKprobe("sys_write")
	if err != nil {
		fmt.Fprintf(os.Stderr, "could not attach kprobe: %v\n", err)
		os.Exit(1)
	}

    // Setup ring buffer
    eventsMap, err := bpfModule.GetMap("events")
    if err != nil {
        fmt.Fprintf(os.Stderr, "could not find events map: %v\n", err)
        os.Exit(1)
    }

    ringBuffer, err := libbpfgo.NewRingBuffer(eventsMap, func(data []byte) {
        // Process each event
        fmt.Println("Event received")
        // Add your logic to process ring buffer events here
    })
    if err != nil {
        fmt.Fprintf(os.Stderr, "could not create ring buffer: %v\n", err)
        os.Exit(1)
    }
    ringBuffer.Start()
    defer ringBuffer.Stop()

    // Wait for the user to press Ctrl+C
    fmt.Println("Waiting for events... Press Ctrl+C to stop.")
    c := make(chan os.Signal, 1)
    signal.Notify(c, os.Interrupt, syscall.SIGTERM)
    <-c
    fmt.Println("Detaching and closing BPF program")
}

Error on command go build -o ./loader ./loader.go

# github.com/aquasecurity/tracee/libbpfgo

../../../go/pkg/mod/github.com/aquasecurity/tracee/[email protected]/libbpfgo.go:418:10: could not determine kind of name for C.bpf_map__resize

../../../go/pkg/mod/github.com/aquasecurity/tracee/[email protected]/libbpfgo.go:315:9: could not determine kind of name for C.bpf_object__open_buffer

../../../go/pkg/mod/github.com/aquasecurity/tracee/[email protected]/libbpfgo.go:636:9: could not determine kind of name for C.bpf_program__set_tracepoint

cgo: 

gcc errors for preamble:

../../../go/pkg/mod/github.com/aquasecurity/tracee/[email protected]/libbpfgo.go: In function 'init_perf_buf':

../../../go/pkg/mod/github.com/aquasecurity/tracee/[email protected]/libbpfgo.go:69:12: error: 'struct perf_buffer_opts' has no member named 'sample_cb'

   69 |     pb_opts.sample_cb = perfCallback;

      |            ^

../../../go/pkg/mod/github.com/aquasecurity/tracee/[email protected]/libbpfgo.go:70:12: error: 'struct perf_buffer_opts' has no member named 'lost_cb'

   70 |     pb_opts.lost_cb = perfLostCallback;

      |            ^

../../../go/pkg/mod/github.com/aquasecurity/tracee/[email protected]/libbpfgo.go:71:12: error: 'struct perf_buffer_opts' has no member named 'ctx'

   71 |     pb_opts.ctx = (void*)ctx;

      |            ^

../../../go/pkg/mod/github.com/aquasecurity/tracee/[email protected]/libbpfgo.go:72:10: error: too few arguments to function 'perf_buffer__new'

   72 |     pb = perf_buffer__new(map_fd, page_cnt, &pb_opts);

      |          ^~~~~~~~~~~~~~~~

In file included from ../../../go/pkg/mod/github.com/aquasecurity/tracee/[email protected]/libbpfgo.go:7:

/usr/include/bpf/libbpf.h:1499:1: note: declared here

 1499 | perf_buffer__new(int map_fd, size_t page_cnt,

From what I saw this seems like some incompatibility between versions, but I reinstalled both go and libbpf 3 times and it didn't work
Thanks!

geyslan · 2024-03-20T00:03:08Z

geyslan
Mar 20, 2024
Maintainer

Hi @Hugo96870, it's nice to see you trying libbpfgo.

We recommend you build it statically since you'll have a different libbpf system version (most of the time).

For building your code correctly, please refer to the selftests Makefile that sets the required flags: https://github.com/aquasecurity/libbpfgo/blob/main/selftest/common/Makefile

0 replies

Hugo96870 · 2024-03-26T21:59:26Z

Hugo96870
Mar 26, 2024
Author

Hi @Hugo96870, it's nice to see you trying libbpfgo.

We recommend you build it statically since you'll have a different libbpf system version (most of the time).

For building your code correctly, please refer to the selftests Makefile that sets the required flags: https://github.com/aquasecurity/libbpfgo/blob/main/selftest/common/Makefile

Hi @geyslan! I'm sorry for the delayed response
Upon build libbpfgo statically a new error started popping up:

go build -o ./trace_write/loader ./trace_write/loader.go

# github.com/aquasecurity/libbpfgo

../libbpfgo/map-common.go:51:39: could not determine kind of name for C.BPF_MAP_TYPE_BLOOM_FILTER

Even if I comment everything related with BPF_MAP_TYPE_BLOOM_FILTER, we get the following error:

go build -o ./trace_write/loader ./trace_write/loader.go

# github.com/aquasecurity/libbpfgo

cgo: ../libbpfgo/prog.go:520:18: unexpected: -1-byte enum type - enum bpf_cgroup_iter_order {}

These are with the changes to use the local repos (example of my go.mod file):

module github.com/cilium/ebpf

go 1.22.0

require (
	github.com/aquasecurity/libbpfgo v0.1.1
	github.com/libbpf/libbpf v1.3.0
	github.com/go-quicktest/qt v1.101.0
	github.com/google/go-cmp v0.5.9
	golang.org/x/exp v0.0.0-20230224173230-c95f2b4c22f2
	golang.org/x/sys v0.17.0
)

replace github.com/aquasecurity/libbpfgo => ../libbpfgo
replace github.com/libbpf/libbpf => ../libbpf

require (
	github.com/kr/pretty v0.3.1 // indirect
	github.com/kr/text v0.2.0 // indirect
	github.com/rogpeppe/go-internal v1.11.0 // indirect
)

If I don't use the local repos the error is the one I started with as expected

0 replies

geyslan · 2024-03-26T22:40:16Z

geyslan
Mar 26, 2024
Maintainer

Check if your vmlinux.h has a definition for BPF_MAP_TYPE_BLOOM_FILTER.

libbpfgo/selftest/common/vmlinux.h

Line 128 in 3c5cd66

BPF_MAP_TYPE_BLOOM_FILTER = 30,

More related:

0 replies

Hugo96870 · 2024-03-30T12:59:46Z

Hugo96870
Mar 30, 2024
Author

Check if your vmlinux.h has a definition for BPF_MAP_TYPE_BLOOM_FILTER.

libbpfgo/selftest/common/vmlinux.h

Line 128 in 3c5cd66

BPF_MAP_TYPE_BLOOM_FILTER = 30,

More related:

I'm facing this issue: could not determine kind of name for C.BPF_MAP_TYPE_BLOOM_FILTER #401

Error compiling go programme #400

Yes the vmlinux.h has a definition for BPF_MAP_TYPE_BLOOM_FILTER
Upon overpassing this issue I'm now facing a new one:

sudo go build -o ./trace_write/loader ./trace_write/loader.go

# github.com/aquasecurity/libbpfgo

cgo: ../libbpfgo/prog.go:520:18: unexpected: -1-byte enum type - enum bpf_cgroup_iter_order {}

This also happens on version of libbpf 1.2 and libbpfgo v0.5.1-libbpf-1.2, from what I could gather the issue is in the struct cgo_bpf_iter_attach_opts_new not being able to recognise bpf_cgroup_iter_order, which is defined in libbpf

I see its definition in here: /<My_path>/libbpfgo/libbpf/.github/actions/build-selftests/vmlinux.h and here:
/home/hugo/Desktop/libbpfgo/libbpf/include/uapi/linux/bpf.h

0 replies

geyslan · 2024-04-01T20:10:39Z

geyslan
Apr 1, 2024
Maintainer

Upon overpassing this issue I'm now facing a new one:

How did you solved it (BPF_MAP_TYPE_BLOOM_FILTER missing def)? I ask since the lack of bpf_cgroup_iter_order seems related to it.

This also happens on version of libbpf 1.2 and libbpfgo v0.5.1-libbpf-1.2

The last libbpfgo release - v0.6.0-libbpf-1.3 - requires libbpf 1.3.

Please, try using the last release and itslibbpf 1.3 submodule. We don't recommend to use the system or other local version.

0 replies

Hugo96870 · 2024-04-01T20:36:54Z

Hugo96870
Apr 1, 2024
Author

How did you solved it (BPF_MAP_TYPE_BLOOM_FILTER missing def)? I ask since the lack of bpf_cgroup_iter_order seems related to it.

With some local changes, but it led to a dead end.

The last libbpfgo release - v0.6.0-libbpf-1.3 - requires libbpf 1.3.

With this version the error that pops is the first one:

/root/go/pkg/mod/github.com/aquasecurity/[email protected]/map-common.go:51:39: could not determine kind of name for C.BPF_MAP_TYPE_BLOOM_FILTER

All the selftests pass and following the other threads led to dead ends as well
Also the vmlinux.h I'm using to compile the ebpf program has the type BPF_MAP_TYPE_BLOOM_FILTER defined in the enum
bpf_map_type

0 replies

geyslan · 2024-04-01T21:49:53Z

geyslan
Apr 1, 2024
Maintainer

Well, I did a local test with your code and realized that your golang import is wrong:

"github.com/aquasecurity/tracee/libbpfgo"

Replace it with:

"github.com/aquasecurity/libbpfgo"

My local test:

::::::::::::::
ebpf.c
::::::::::::::
// +build ignore

#include <vmlinux.h>
#include <bpf/bpf_helpers.h>

// Define a map to hold the count of bytes written
struct {
    __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
    __type(key, u32);
    __type(value, long);
    __uint(max_entries, 64);
} write_count SEC(".maps");

SEC("kprobe/sys_write")
int bpf_prog1(const struct pt_regs *ctx)
{
    u32 key = 0;
    long bytes_written;

    bpf_probe_read(&bytes_written, sizeof(bytes_written), (void *)ctx + 24);

    // Update the map with the count of bytes written
    bpf_map_update_elem(&write_count, &key, &bytes_written, BPF_ANY);

    return 0;
}

char LICENSE[] SEC("license") = "GPL";
::::::::::::::
loader.go
::::::::::::::
package main

import (
    "fmt"
    "os"
    "os/signal"
    "syscall"
    "github.com/aquasecurity/libbpfgo"
)

func main() {
    // Load the BPF Object
    bpfModule, err := libbpfgo.NewModuleFromFile("write_trace.bpf.o")
    if err != nil {
        fmt.Fprintf(os.Stderr, "could not load bpf module: %v\n", err)
        os.Exit(1)
    }
    defer bpfModule.Close()

    // Load and attach tracepoint
    err = bpfModule.BPFLoadObject()
    if err != nil {
        fmt.Fprintf(os.Stderr, "could not load bpf object: %v\n", err)
        os.Exit(1)
    }

	prog, err := bpfModule.GetProgram("bpf_prog1")
	if err != nil {
		fmt.Fprintf(os.Stderr, "could not get BPF program: %v\n", err)
		os.Exit(1)
	}

	_, err = prog.AttachKprobe("sys_write")
	if err != nil {
		fmt.Fprintf(os.Stderr, "could not attach kprobe: %v\n", err)
		os.Exit(1)
	}

    // Setup ring buffer
    eventsMap, err := bpfModule.GetMap("events")
    if err != nil {
        fmt.Fprintf(os.Stderr, "could not find events map: %v\n", err)
        os.Exit(1)
    }

    ringBuffer, err := libbpfgo.NewRingBuffer(eventsMap, func(data []byte) {
        // Process each event
        fmt.Println("Event received")
        // Add your logic to process ring buffer events here
    })
    if err != nil {
        fmt.Fprintf(os.Stderr, "could not create ring buffer: %v\n", err)
        os.Exit(1)
    }
    ringBuffer.Start()
    defer ringBuffer.Stop()

    // Wait for the user to press Ctrl+C
    fmt.Println("Waiting for events... Press Ctrl+C to stop.")
    c := make(chan os.Signal, 1)
    signal.Notify(c, os.Interrupt, syscall.SIGTERM)
    <-c
    fmt.Println("Detaching and closing BPF program")
}
::::::::::::::
go.mod
::::::::::::::
module testbpf.com

go 1.22.0

require github.com/aquasecurity/libbpfgo v0.6.0-libbpf-1.3

replace github.com/aquasecurity/libbpfgo => ./libbpfgo
::::::::::::::
go.sum
::::::::::::::
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

Command to compile it:

GOOS=linux CC=clang GOARCH=amd64 CGO_CFLAGS="-I ./libbpfgo/output -I ./libbpfgo/selftest/common" CGO_LDFLAGS="$(pkg-config --libs libelf zlib) ./libbpfgo/output/libbpf/libbpf.a" go build -ldflags='-w -extldflags "-static"' -o loader

It outputs other error since you don't have a BPF_MAP_TYPE_RINGBUF.

0 replies

geyslan · 2024-04-01T21:51:00Z

geyslan
Apr 1, 2024
Maintainer

I'm going to convert it to a discussion since we didn't detect any issue.

0 replies

Hugo96870 · 2024-04-02T23:22:23Z

Hugo96870
Apr 2, 2024
Author

We can continue this as a discussion yes!

I just replaced evertything with the programs you specified and the first error still persists:

GOOS=linux CC=clang GOARCH=amd64 CGO_CFLAGS="-I ./libbpfgo/output -I ./libbpfgo/selftest/common" CGO_LDFLAGS="$(pkg-config --libs libelf zlib) ./libbpfgo/output/libbpf/libbpf.a" go build -ldflags='-w -extldflags "-static"' -o loader

# github.com/aquasecurity/libbpfgo

../../../go/pkg/mod/github.com/aquasecurity/[email protected]/map-common.go:51:39: could not determine kind of name for C.BPF_MAP_TYPE_BLOOM_FILTER

1 reply

geyslan Sep 6, 2024
Maintainer

@Hugo96870 sorry for the lateness. I suspect you already have solved this. Anyway check #401 (reply in thread).

Thanks.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Incompatability between libbpf and libbpfgo #413

{{title}}

Replies: 9 comments 1 reply

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Incompatability between libbpf and libbpfgo #413

Hugo96870 Mar 19, 2024

Replies: 9 comments · 1 reply

geyslan Mar 20, 2024 Maintainer

Hugo96870 Mar 26, 2024 Author

geyslan Mar 26, 2024 Maintainer

Hugo96870 Mar 30, 2024 Author

geyslan Apr 1, 2024 Maintainer

Hugo96870 Apr 1, 2024 Author

geyslan Apr 1, 2024 Maintainer

geyslan Apr 1, 2024 Maintainer

Hugo96870 Apr 2, 2024 Author

geyslan Sep 6, 2024 Maintainer

Hugo96870
Mar 19, 2024

Replies: 9 comments 1 reply

geyslan
Mar 20, 2024
Maintainer

Hugo96870
Mar 26, 2024
Author

geyslan
Mar 26, 2024
Maintainer

Hugo96870
Mar 30, 2024
Author

geyslan
Apr 1, 2024
Maintainer

Hugo96870
Apr 1, 2024
Author

geyslan
Apr 1, 2024
Maintainer

geyslan
Apr 1, 2024
Maintainer

Hugo96870
Apr 2, 2024
Author

geyslan Sep 6, 2024
Maintainer