Merge pull request #216 from IntelLabs/docs/kernel_tutorial_ip_ranges…

…_implicit Docs: Update Linux kernel tutorial to use implicit IP parameters from state.yaml snapshot file
IntelLabs · Aug 21, 2023 · fad51da · fad51da
2 parents 17abced + ef7abf4
commit fad51da
Show file tree

Hide file tree

Showing 2 changed files with 19 additions and 41 deletions.
diff --git a/.github/RELEASE.md b/.github/RELEASE.md
@@ -4,16 +4,21 @@
 
 # ✨ Improvements
 
-/
+- Fuzzer / QEMU:
+  Dump snapshot metadata into `$WORKDIR/snapshot/state.yml`, and parse that file on fuzzer shutdown to update it's own IP filters configuration. (see [`kafl.fuzzer#68`](https://github.com/IntelLabs/kafl.fuzzer/pull/68), [`kafl.qemu#10`](https://github.com/IntelLabs/kafl.qemu/pull/10))
+
+  This avoids having to pass IP filters through the `hprintf` channel, parse logs and extract them on the host, to send them again to `kafl cov` for coverage.
 
 # 🔧 Fixes
 
 - Security fixes (#215, #217)
 - Switch QEMU revision pinning to a tag instead of a branch (#214)
+- `kafl.targets`: simplify linux kernel tutorial and use predefined load path for kAFL configuration ([`kafl.targets#23`](https://github.com/IntelLabs/kafl.targets/pull/23))
 
 # 📖 Documentation
 
 - Misc fixes (#213)
+- Linux kernel tutorial: use implicit IP filters from snapshot metadata: (https://intellabs.github.io/kAFL/reference/hypercall_api.html#range-submit) (#216)
 
 
 # 🧰 Behind the scenes

diff --git a/docs/source/tutorials/fuzzing_linux_kernel.md b/docs/source/tutorials/fuzzing_linux_kernel.md
@@ -48,10 +48,10 @@ launching the fuzzer is as simple as booting the kernel:
 ::::{tab-set}
 :::{tab-item} Local setup
 ```shell
-KAFL_CONFIG_FILE=./kafl_config.yaml kafl fuzz --purge \
-	--redqueen --grimoire -D --radamsa \
-	--kernel linux-guest/arch/x86/boot/bzImage \
-	-t 0.1 -ts 0.01 -m 512 --log-crashes -p 2
+kafl fuzz --purge \
+        --redqueen --grimoire -D --radamsa \
+        --kernel linux-guest/arch/x86/boot/bzImage \
+        -t 0.1 -ts 0.01 -m 512 --log-crashes -p 2
 ```
 :::
 :::{tab-item} Docker image
@@ -62,10 +62,10 @@ mkdir -p /dev/shm/kafl_$(id -un)
 docker run \
         -ti --rm \
         --device /dev/kvm \
-        -v $(pwd)/kafl_config.yaml:/mnt/kafl_config.yaml \
+        -v $(pwd)/kafl.yaml:/mnt/kafl.yaml \
         -v /dev/shm/kafl_$(id -un):/mnt/workdir \
         -v $(pwd)/linux-guest/arch/x86/boot/bzImage:/mnt/kernel \
-        -e KAFL_CONFIG_FILE=/mnt/kafl_config.yaml \
+        -e KAFL_CONFIG_FILE=/mnt/kafl.yaml \
         --user $(id -u):$(id -g) \
         --group-add $(getent group kvm | cut -d: -f3) \
         intellabs/kafl \
@@ -176,49 +176,24 @@ ls $KAFL_WORKDIR/logs/
 
 ## 5. Coverage
 
-For coverage reports, we need to first find out what PT filter ranges are used for the kernel image.
-They are auto-detected on startup and can be logged using `--log-hprintf` parameter.
-
-Try to launch with `--log-hprintf -p 1` instead of the above `--log-crashes` and look at `$KAFL_WORKDIR/hprintf_00.log`.
-
-Sample content for `KAFL_WORKDIR/hprintf_00.log`:
-```
-...
-Submitting payload buffer address to hypervisor (ffffffff859ad000)
-Setting range 0: ffffffff81000000-ffffffff83603000
-Setting range 1: ffffffff855ed000-ffffffff856e4000
-Starting kAFL loop...
-...
-```
-
-Lets extract these IP ranges, and export them in environment variables:
-
-```shell
-export IP0=$(grep -oP 'Setting range 0:\s+\K.*-.*' $KAFL_WORKDIR/hprintf_00.log)
-export IP1=$(grep -oP 'Setting range 1:\s+\K.*-.*' $KAFL_WORKDIR/hprintf_00.log)
-```
+To gather the coverage, use `kafl cov` subcommand with `--resume`. This will reload the guest state directly from the Nyx fast-snapshot used during fuzzing and re-use the existing `$KAFL_WORKDIR/page_cache*` files, leading to better reproducibility.
 
-Then use `kafl cov` subcommand with `--resume` will reload the guest state directly from the Nyx fast-snapshot used during fuzzing and re-use the existing `$KAFL_WORKDIR/page_cache*` files, leading to better reproducibility.
+PT traces produced by QEMU/worker instances are picked up from `$KAFL_WORKDIR/pt_trace_dump_NN` and stored at `$KAFL_WORKDIR/traces/*bin.lz4`.
 
-PT traces produced by Qemu/worker instances are
-picked up from `$KAFL_WORKDIR/pt_trace_dump_NN` and stored at `$KAFL_WORKDIR/traces/*bin.lz4`.
 The `kafl cov` tool then calls `ptdump` with the given PT filter range and
 `page_cache` files to decode to a corresponding text file `$KAFL_WORKDIR/traces/*.txt.lz4`.
 
 For best results, it is recommended to collect binary PT traces already during
-fuzzing (using `kafl fuzz --trace` option). The `kafl cov` tool will detect the
-existing binary traces in `$KAFL_WORKDIR/traces/` and skip re-executing the corpus,
-providing accurate coverage traces even for non-deterministic targets.
+fuzzing (using `kafl fuzz [--trace](../reference/fuzzer_configuration.md#trace)` option). The `kafl cov` tool will detect the
+existing binary traces in `$KAFL_WORKDIR/traces/` and skip re-executing the corpus, providing accurate coverage traces even for non-deterministic targets.
 
 For big corpuses, you can parallelize this process using `-p`:
 
 ::::{tab-set}
 :::{tab-item} Local setup
 ```shell
-KAFL_CONFIG_FILE=kafl_config.yaml kafl cov \
+kafl cov \
 	--kernel linux-guest/arch/x86/boot/bzImage \
-	-ip0 $IP0 \
-	-ip1 $IP1 \
 	--resume -m 512 -t 2 -p 24
 ```
 :::
@@ -227,19 +202,17 @@ KAFL_CONFIG_FILE=kafl_config.yaml kafl cov \
 docker run \
         -ti --rm \
         --device /dev/kvm \
-        -v $(pwd)/kafl_config.yaml:/mnt/kafl_config.yaml \
+        -v $(pwd)/kafl.yaml:/mnt/kafl.yaml \
         -v /dev/shm/kafl_$(id -un):/mnt/workdir \
         -v $(pwd)/linux-guest/arch/x86/boot/bzImage:/mnt/kernel \
-        -e KAFL_CONFIG_FILE=/mnt/kafl_config.yaml \
+        -e KAFL_CONFIG_FILE=/mnt/kafl.yaml \
         --user $(id -u):$(id -g) \
         --group-add $(getent group kvm | cut -d: -f3) \
         intellabs/kafl \
         cov \
         -w /mnt/workdir \
         --input /mnt/workdir \
         --kernel /mnt/kernel \
-        -ip0 $IP0 \
-        -ip1 $IP1 \
         --resume -m 512 -t 2 -p 24
 ```
 :::