CI: add CI for runtime tests #338

yunwei37 · 2024-08-21T02:39:08Z

Description

Fixes #332

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)

hp77-creator · 2024-08-21T14:22:27Z

@yunwei37 can we also add macOS machine tests in this?
Also for the code coverage, I have shared a secret with you in personal chat, please add those then I can configure codecov

yunwei37 · 2024-08-21T20:36:50Z

@hp77-creator Can you help fix the build in github actions? We can test in Linux first.

yunwei37 · 2024-08-21T20:41:11Z

Seems there are github actions that can run on MacOS: https://github.com/actions/runner-images/tree/main/images/macos

You can pick one and try to add them.

yunwei37 · 2024-08-21T20:43:09Z

I also create a issue in #340

hp77-creator · 2024-08-22T15:11:17Z

Shall I add my changes in this PR only @yunwei37 ?

hp77-creator · 2024-08-22T15:11:29Z

Seems there are github actions that can run on MacOS: https://github.com/actions/runner-images/tree/main/images/macos

You can pick one and try to add them.

I will check these.

yunwei37 · 2024-08-22T19:36:41Z

You can add change to this pr by pushing to the branch. (I think you have access to that?

Thanks!

hp77-creator · 2024-08-23T18:00:02Z

very strange thing is happening, I tried running the commands that you have used for without-libbpf
It is working fine in both MacOS and Linux on MacOS.
I think this is some dependency from the errors but I see that you have not changed any configuration as such. I will check.

yunwei37 · 2024-08-23T21:37:09Z

Maybe you have installed libbpf locally in Linux? I also get similar results before

hp77-creator · 2024-08-24T02:50:03Z

Maybe you have installed libbpf locally in Linux? I also get similar results before

yea, so on my VM, there is already libbpf present, so its working fine, on MacOS, that is not required so its not using it. Hence tests are passing on my systems but are failing in CI. I will add the guards where there is use of libbpf

hp77-creator · 2024-08-24T06:32:09Z

Fedora is working now but Ubuntu is failing because bpf_attr struct doesn't have map_extra in the ubuntu version.
I checked on my system for Ubuntu:22.04 and could see the same, Is there any way we can add the updated <linux/bpf.h> file in ubuntu?

hp77-creator

LGTM. @Officeyutong can you also check once on the guards that I have added.

Officeyutong · 2024-08-24T16:12:02Z

.github/workflows/test-vm.yml

@@ -17,7 +17,7 @@ jobs:
          - ubuntu-2204
          - fedora-39
    container: 
-      image: "manjusakalza/bpftime-base-image:${{matrix.container}}"
+      image: "hp77creator/bpftime-base-image:${{matrix.container}}"


What about getting these images to account eunomia-bpf? Do you have permission to do that?

for the ghci images, will have to configure i guess, For the time being, I focussed more on making the existing setup pass. We can create an issue to migrate these to eunomia ones. Works @Officeyutong ?

#341 addresses this. You can add more requirements on the issue if I am missing something.

runtime/syscall-server/syscall_context.cpp

Officeyutong · 2024-08-24T17:20:17Z

runtime/syscall-server/syscall_context.cpp

+#if __linux__ && !BPFTIME_BUILD_WITH_LIBBPF
+#define offsetofend(type, member) (offsetof(type, member) + sizeof(((type *)0)->member))
+static void *libc_handle = dlopen(LIBC_SO, RTLD_LAZY);
+static auto libc_syscall =


Original syscall from libc was already stored at

bpftime/runtime/syscall-server/syscall_context.hpp

Line 92 in b0ac6d9

syscall_fn orig_syscall_fn = nullptr;

above definition is to support the bpf_get_obj_by_fd function. I see that there is a place where original syscall is being stored, but it is defined inside a member function. Do you want me to replace libc_handle with syscall_fun ?

Where was bpf_get_obj_by_fd called?

wrong function at my end. It was actually bpf_obj_get_info_by_fd

bpf_obj_get_info_by_fd

bpf_obj_get_info_by_fd is only called by member function of syscall_context, so it's ok to use functions stored in syscall_context

hp77-creator · 2024-08-30T03:10:37Z

@yunwei37 @Officeyutong build is failing now becaues of this:
gmake[3]: *** [vm/compat/llvm-vm/llvm-jit/CMakeFiles/llvmbpf_vm.dir/build.make:90: vm/compat/llvm-vm/llvm-jit/CMakeFiles/llvmbpf_vm.dir/src/compiler.cpp.o] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:958: vm/compat/llvm-vm/llvm-jit/CMakeFiles/llvmbpf_vm.dir/all] Error 2
gmake[2]: *** Waiting for unfinished jobs....

any idea?

I have run the exact cmake commands in my fedora VM, its building fine there

Officeyutong · 2024-08-30T08:12:50Z

@yunwei37 @Officeyutong build is failing now becaues of this: gmake[3]: *** [vm/compat/llvm-vm/llvm-jit/CMakeFiles/llvmbpf_vm.dir/build.make:90: vm/compat/llvm-vm/llvm-jit/CMakeFiles/llvmbpf_vm.dir/src/compiler.cpp.o] Error 1 gmake[2]: *** [CMakeFiles/Makefile2:958: vm/compat/llvm-vm/llvm-jit/CMakeFiles/llvmbpf_vm.dir/all] Error 2 gmake[2]: *** Waiting for unfinished jobs....

any idea?

I have run the exact cmake commands in my fedora VM, its building fine there

Quote reply

It fails because of another recent PR, which changed the definition of struct ebpf_inst, the exact error is at https://github.com/eunomia-bpf/bpftime/actions/runs/10626245841/job/29457492213?pr=338#step:4:336

Field off was renamed to offset by eunomia-bpf/llvmbpf#9

You may need to update submodule, since llvmbpf is now in a submodule

hp77-creator · 2024-08-30T19:42:50Z

Now more things are breaking, somewhere code is being used instead of opcode, somewhere some virtual are used in place of optional, Which version to upgrade it too? @yunwei37 @Officeyutong

yunwei37 · 2024-08-30T20:53:05Z

No worries, I will help fix that

hp77-creator · 2024-09-05T19:15:02Z

No worries, I will help fix that

@yunwei37 for that CI codecov, shall I raise one or will this be fixed before?

codecov · 2024-09-30T18:14:19Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (master@0d8402e). Learn more about missing BASE report.

Additional details and impacted files

@@            Coverage Diff            @@
##             master     #338   +/-   ##
=========================================
  Coverage          ?   97.34%           
=========================================
  Files             ?       10           
  Lines             ?      377           
  Branches          ?       12           
=========================================
  Hits              ?      367           
  Misses            ?       10           
  Partials          ?        0

Flag	Coverage Δ
attach tests (uprobe & syscall trace)	`97.34% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

* add xdp-counter example * add build in ci

* fix link dl and cmake vesion * add dev container * docs: include PATH in the README * bump upload to v4 * fix hidden files

* feat:add lcov support for test * update * update * update * update * update * update * update * update * update * update * update

pull-request-size bot added the size/S label Aug 21, 2024

yunwei37 requested a review from hp77-creator August 21, 2024 04:21

pull-request-size bot added size/M and removed size/S labels Aug 21, 2024

hp77-creator force-pushed the fix-runtime-ci branch from 56b5db0 to ed9212a Compare August 23, 2024 19:18

pull-request-size bot added size/L size/M and removed size/M size/L labels Aug 24, 2024

hp77-creator force-pushed the fix-runtime-ci branch from 0908018 to 9c9a7d9 Compare August 24, 2024 15:15

pull-request-size bot added size/L and removed size/M labels Aug 24, 2024

hp77-creator approved these changes Aug 24, 2024

View reviewed changes

Officeyutong reviewed Aug 24, 2024

View reviewed changes

hp77-creator mentioned this pull request Aug 24, 2024

[FEATURE] Migrate workflows which use docker-hub images to ghci images #341

Open

Officeyutong requested changes Aug 24, 2024

View reviewed changes

pull-request-size bot added size/M and removed size/L labels Aug 30, 2024

update CI sripts

3d20f7f

hp77-creator added 4 commits August 30, 2024 08:14

Revert container images

c0c3413

Add linux kernel guards

6624e75

Fix guard

4d4bd53

remove new syscall variable

962508a

hp77-creator force-pushed the fix-runtime-ci branch from f83a86d to 962508a Compare August 30, 2024 02:46

pull-request-size bot added size/L and removed size/M labels Aug 30, 2024

yunwei37 force-pushed the master branch 3 times, most recently from 1354316 to a6a132b Compare September 11, 2024 18:08

yunwei37 force-pushed the fix-runtime-ci branch from d91c367 to 47f7087 Compare September 30, 2024 18:11

pull-request-size bot added size/S and removed size/L labels Sep 30, 2024

yunwei37 and others added 8 commits September 30, 2024 18:24

add xdp-counter example (#339)

1e5dab7

* add xdp-counter example * add build in ci

fix: compile on old cmake version and add PATH (#346)

9f6f027

* fix link dl and cmake vesion * add dev container * docs: include PATH in the README * bump upload to v4 * fix hidden files

Docs: update readme ffor network

ddbf20a

Docs: rewrite Components and Design

68b6b89

docs: add notice (#349)

30a993e

feat:add lcov and codecov support (#352)

4ccf5b3

* feat:add lcov support for test * update * update * update * update * update * update * update * update * update * update * update

fix runtime ci

af53f97

update readme

d3993e1

yunwei37 force-pushed the fix-runtime-ci branch from 47f7087 to d3993e1 Compare September 30, 2024 18:26

pull-request-size bot added size/XL and removed size/S labels Sep 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: add CI for runtime tests #338

CI: add CI for runtime tests #338

yunwei37 commented Aug 21, 2024

hp77-creator commented Aug 21, 2024

yunwei37 commented Aug 21, 2024

yunwei37 commented Aug 21, 2024

yunwei37 commented Aug 21, 2024

hp77-creator commented Aug 22, 2024

hp77-creator commented Aug 22, 2024

yunwei37 commented Aug 22, 2024

hp77-creator commented Aug 23, 2024

yunwei37 commented Aug 23, 2024

hp77-creator commented Aug 24, 2024

hp77-creator commented Aug 24, 2024

hp77-creator left a comment

Officeyutong Aug 24, 2024

hp77-creator Aug 24, 2024

hp77-creator Aug 24, 2024

Officeyutong Aug 24, 2024

hp77-creator Aug 25, 2024

Officeyutong Aug 25, 2024

hp77-creator Aug 25, 2024

Officeyutong Aug 25, 2024

hp77-creator commented Aug 30, 2024 •

edited

Loading

Officeyutong commented Aug 30, 2024 •

edited

Loading

hp77-creator commented Aug 30, 2024

yunwei37 commented Aug 30, 2024

hp77-creator commented Sep 5, 2024

codecov bot commented Sep 30, 2024 •

edited

Loading

CI: add CI for runtime tests #338

Are you sure you want to change the base?

CI: add CI for runtime tests #338

Conversation

yunwei37 commented Aug 21, 2024

Description

Type of change

hp77-creator commented Aug 21, 2024

yunwei37 commented Aug 21, 2024

yunwei37 commented Aug 21, 2024

yunwei37 commented Aug 21, 2024

hp77-creator commented Aug 22, 2024

hp77-creator commented Aug 22, 2024

yunwei37 commented Aug 22, 2024

hp77-creator commented Aug 23, 2024

yunwei37 commented Aug 23, 2024

hp77-creator commented Aug 24, 2024

hp77-creator commented Aug 24, 2024

hp77-creator left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hp77-creator commented Aug 30, 2024 • edited Loading

Officeyutong commented Aug 30, 2024 • edited Loading

hp77-creator commented Aug 30, 2024

yunwei37 commented Aug 30, 2024

hp77-creator commented Sep 5, 2024

codecov bot commented Sep 30, 2024 • edited Loading

Codecov Report

hp77-creator commented Aug 30, 2024 •

edited

Loading

Officeyutong commented Aug 30, 2024 •

edited

Loading

codecov bot commented Sep 30, 2024 •

edited

Loading