libFuzzer continuous fuzzing/jobs

[GrosQuildu]

libFuzzer stops when first crash is found. We may change that by providing -jobs=9999 option, which may be good for continuous fuzzing. But restart may be better?

Anyway, once the option is used, libFuzzer stops printing statistics to stdout and writes them to fuzz-JOB.txt file, which position (in filesystem) we can't control.

Research about libFuzzer continuous fuzzing is required. Then appropriate fixes, including changes to populate_stats method.

[Travmatth]

I've tried reproducing this issue and I'm not sure what the actual vs. intended behavior should be:

user@bb2e0f74dd7a:~/deepstate$ cat examples/MultiCrash.cpp
#include <deepstate/DeepState.hpp>

using namespace deepstate;

DEEPSTATE_NOINLINE static char* segfault(char *x) {
  if (x[0] == '\xde') {
    if (x[1] == '\xad') {
      unsigned *p = NULL;
      *(p+1) = 0xdeadbeef;  // Trigger segfault here
    }
  }
  if (x[0] == '\xbe') {
    if (x[1] == '\xef') {
      unsigned *p = NULL;
      *(p+1) = 0xdeadbeef;  // Trigger segfault here
    }
  }
  return x;
}

TEST(SimpleCrash, SegFault) {
  char *x = DeepState_CStr_C(4, 0);

  segfault(x);

  ASSERT_EQ(x, x);
}

user@bb2e0f74dd7a:~/deepstate$ deepstate-libfuzzer --compile_test examples/MultiCrash.cpp --out_test_name MultiCrash
INFO:deepstate:Setting log level from DEEPSTATE_LOG: 2
INFO:deepstate.core.base:Setting log level from --min_log_level: 2
INFO:deepstate.core.fuzz:Calling pre_exec before fuzzing
INFO:deepstate.core.fuzz:Compiling test harness `['/usr/bin/clang++', '-std=c++11', 'examples/MultiCrash.cpp', '-ldeepstate_LF', '-fsanitize=fuzzer,undefined', '-o', 'MultiCrash.libfuzzer']`

Done compiling target binary `/home/user/deepstate/MultiCrash.libfuzzer`.

user@bb2e0f74dd7a:~/deepstate$ mkdir libfuzzer_out

user@bb2e0f74dd7a:~/deepstate$ deepstate-libfuzzer --output_test_dir libfuzzer_out ./MultiCrash.libfuzzer
INFO:deepstate:Setting log level from DEEPSTATE_LOG: 2
INFO:deepstate.core.base:Setting log level from --min_log_level: 2
INFO:deepstate.core.fuzz:Calling pre_exec before fuzzing
INFO:deepstate.core.fuzz:Executing command `['/home/user/deepstate/MultiCrash.libfuzzer', '-rss_limit_mb=50', '-max_len=8192', '-artifact_prefix=libfuzzer_out/the_fuzzer/crashes/', '-reload=1', '-runs=-1', '-print_final_stats=1', 'libfuzzer_out/sync_dir/queue']`
INFO:deepstate.core.fuzz:Using DeepState output.
INFO:deepstate.core.fuzz:Started fuzzer process with PID 160.
INFO:deepstate.core.fuzz:Fuzzer libFuzzer (PID 160) exited with return code 1.
FUZZ_STATS:deepstate.core.fuzz:unique_crashes:1
FUZZ_STATS:deepstate.core.fuzz:fuzzer_pid:160
FUZZ_STATS:deepstate.core.fuzz:start_time:1584394713
FUZZ_STATS:deepstate.core.fuzz:------------------------------
INFO:deepstate.core.fuzz:Killing process 160 and childs.
INFO:deepstate.core.fuzz:Using DeepState output.
INFO:deepstate.core.fuzz:Started fuzzer process with PID 163.
INFO:deepstate.core.fuzz:Fuzzer libFuzzer (PID 163) exited with return code 1.
FUZZ_STATS:deepstate.core.fuzz:unique_crashes:2
FUZZ_STATS:deepstate.core.fuzz:fuzzer_pid:163
FUZZ_STATS:deepstate.core.fuzz:start_time:1584394713
FUZZ_STATS:deepstate.core.fuzz:------------------------------
^C

user@bb2e0f74dd7a:~/deepstate$ xxd libfuzzer_out/the_fuzzer/crashes/crash-83a8cb067bc53c39a65fe89bdfdb6e317ce93288
00000000: beef feff ff                             .....

user@bb2e0f74dd7a:~/deepstate$ xxd libfuzzer_out/the_fuzzer/crashes/crash-92ae4452d2a6f076780acf02e45dc153b6751d10
00000000: dead ad53 0753 53

It seems to be finding multiple crashes just fine. Was this fixed in some previous code change, or am I misunderstanding the issue at hand?

[GrosQuildu]

The fuzzer is restarted after the first crash ATM. The issue is mainly to investigate and handle appropriately libfuzzers args like jobs, workers and fork. Cause I wasn't sure how to handle libfuzzer properly and if it will behave correctly in all cases (like what if crashing input is in the input directory, infinite restart loop? etc).