Home | History | Annotate | Line # | Download | only in fuzz
      1 Fuzzing OpenSSL
      2 ===============
      3 
      4 OpenSSL can use either LibFuzzer or AFL to do fuzzing.
      5 
      6 LibFuzzer
      7 ---------
      8 
      9 How to fuzz OpenSSL with [libfuzzer](http://llvm.org/docs/LibFuzzer.html),
     10 starting from a vanilla+OpenSSH server Ubuntu install.
     11 
     12 With `clang` from a package manager
     13 -----------------------------------
     14 
     15 Install `clang`, which [ships with `libfuzzer`](http://llvm.org/docs/LibFuzzer.html#fuzzer-usage)
     16 since version 6.0:
     17 
     18     sudo apt-get install clang
     19 
     20 Configure `openssl` for fuzzing. For now, you'll still need to pass in the path
     21 to the `libFuzzer` library file while configuring; this is represented as
     22 `$PATH_TO_LIBFUZZER` below. A typical value would be
     23 `/usr/lib/llvm-7/lib/clang/7.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a`.
     24 
     25     CC=clang ./config enable-fuzz-libfuzzer \
     26             --with-fuzzer-lib=$PATH_TO_LIBFUZZER \
     27             -DPEDANTIC enable-asan enable-ubsan no-shared \
     28             -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION \
     29             -fsanitize=fuzzer-no-link \
     30             enable-ec_nistp_64_gcc_128 -fno-sanitize=alignment \
     31             enable-weak-ssl-ciphers enable-rc5 enable-md2 \
     32             enable-ssl3 enable-ssl3-method enable-nextprotoneg \
     33             --debug
     34 
     35 Clang uses the gcc libstdc++ library so this must also be installed. You can
     36 check which version of gcc clang is using like this:
     37 
     38     $ clang --verbose
     39     Ubuntu clang version 14.0.0-1ubuntu1.1
     40     Target: x86_64-pc-linux-gnu
     41     Thread model: posix
     42     InstalledDir: /usr/bin
     43     Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/12
     44     Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/10
     45     Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11
     46     Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12
     47     Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12
     48     Candidate multilib: .;@m64
     49     Selected multilib: .;@m64
     50 
     51 So, in the above example clang is using gcc version 12. Ensure that the selected
     52 gcc version has the relevant libstdc++ files installed:
     53 
     54     $ ls /usr/lib/gcc/x86_64-linux-gnu/12 | grep stdc++
     55     libstdc++.a
     56     libstdc++fs.a
     57     libstdc++.so
     58 
     59 On Ubuntu for gcc-12 this requires the libstdc++-12-dev package installed.
     60 
     61     $ sudo apt-get install libstdc++-12-dev
     62 
     63 Compile:
     64 
     65     sudo apt-get install make
     66     make clean
     67     LDCMD=clang++ make -j4
     68 
     69 Finally, perform the actual fuzzing:
     70 
     71     fuzz/helper.py $FUZZER
     72 
     73 where $FUZZER is one of the executables in `fuzz/`.
     74 It will run until you stop it.
     75 
     76 If you get a crash, you should find a corresponding input file in
     77 `fuzz/corpora/$FUZZER-crash/`.
     78 
     79 With `clang` from source/pre-built binaries
     80 -------------------------------------------
     81 
     82 You may also wish to use a pre-built binary from the [LLVM Download
     83 site](http://releases.llvm.org/download.html), or to [build `clang` from
     84 source](https://clang.llvm.org/get_started.html). After adding `clang` to your
     85 path and locating the `libfuzzer` library file, the procedure for configuring
     86 fuzzing is the same, except that you also need to specify
     87 a `--with-fuzzer-include` option, which should be the parent directory of the
     88 prebuilt fuzzer library. This is represented as `$PATH_TO_LIBFUZZER_DIR` below.
     89 
     90     CC=clang ./config enable-fuzz-libfuzzer \
     91             --with-fuzzer-include=$PATH_TO_LIBFUZZER_DIR \
     92             --with-fuzzer-lib=$PATH_TO_LIBFUZZER \
     93             -DPEDANTIC enable-asan enable-ubsan no-shared \
     94             -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION \
     95             -fsanitize=fuzzer-no-link \
     96             enable-ec_nistp_64_gcc_128 -fno-sanitize=alignment \
     97             enable-weak-ssl-ciphers enable-rc5 enable-md2 \
     98             enable-ssl3 enable-ssl3-method enable-nextprotoneg \
     99             --debug
    100 
    101 AFL
    102 ---
    103 
    104 This is an alternative to using LibFuzzer.
    105 
    106 Configure for fuzzing:
    107 
    108     sudo apt-get install afl-clang
    109     CC=afl-clang-fast ./config enable-fuzz-afl no-shared no-module \
    110         -DPEDANTIC enable-tls1_3 enable-weak-ssl-ciphers enable-rc5 \
    111         enable-md2 enable-ssl3 enable-ssl3-method enable-nextprotoneg \
    112         enable-ec_nistp_64_gcc_128 -fno-sanitize=alignment \
    113         --debug
    114     make clean
    115     make
    116 
    117 The following options can also be enabled: enable-asan, enable-ubsan, enable-msan
    118 
    119 Run one of the fuzzers:
    120 
    121     afl-fuzz -i fuzz/corpora/$FUZZER -o fuzz/corpora/$FUZZER/out fuzz/$FUZZER
    122 
    123 Where $FUZZER is one of the executables in `fuzz/`.
    124 
    125 Reproducing issues
    126 ------------------
    127 
    128 If a fuzzer generates a reproducible error, you can reproduce the problem using
    129 the fuzz/*-test binaries and the file generated by the fuzzer. They binaries
    130 don't need to be built for fuzzing, there is no need to set CC or the call
    131 config with enable-fuzz-* or -fsanitize-coverage, but some of the other options
    132 above might be needed. For instance the enable-asan or enable-ubsan option might
    133 be useful to show you when the problem happens. For the client and server fuzzer
    134 it might be needed to use -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION to
    135 reproduce the generated random numbers.
    136 
    137 To reproduce the crash you can run:
    138 
    139     fuzz/$FUZZER-test $file
    140 
    141 To do all the tests of a specific fuzzer such as asn1 you can run
    142 
    143     fuzz/asn1-test fuzz/corpora/asn1
    144 or
    145     make test TESTS=fuzz_test_asn1
    146 
    147 To run several fuzz tests you can use for instance:
    148 
    149     make test TESTS='test_fuzz_cmp test_fuzz_cms'
    150 
    151 To run all fuzz tests you can use:
    152 
    153     make test TESTS='test_fuzz_*'
    154 
    155 Random numbers
    156 --------------
    157 
    158 The client and server fuzzer normally generate random numbers as part of the TLS
    159 connection setup. This results in the coverage of the fuzzing corpus changing
    160 depending on the random numbers. This also has an effect for coverage of the
    161 rest of the test suite and you see the coverage change for each commit even when
    162 no code has been modified.
    163 
    164 Since we want to maximize the coverage of the fuzzing corpus, the client and
    165 server fuzzer will use predictable numbers instead of the random numbers. This
    166 is controlled by the FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION define.
    167 
    168 The coverage depends on the way the numbers are generated. We don't disable any
    169 check of hashes, but the corpus has the correct hash in it for the random
    170 numbers that were generated. For instance the client fuzzer will always generate
    171 the same client hello with the same random number in it, and so the server, as
    172 emulated by the file, can be generated for that client hello.
    173 
    174 Coverage changes
    175 ----------------
    176 
    177 Since the corpus depends on the default behaviour of the client and the server,
    178 changes in what they send by default will have an impact on the coverage. The
    179 corpus will need to be updated in that case.
    180 
    181 Updating the corpus
    182 -------------------
    183 
    184 The client and server corpus is generated with multiple config options:
    185 
    186 - The options as documented above
    187 - Without enable-ec_nistp_64_gcc_128 and without --debug
    188 - With no-asm
    189 - Using 32 bit
    190 - A default config, plus options needed to generate the fuzzer.
    191 
    192 The libfuzzer merge option is used to add the additional coverage
    193 from each config to the minimal set.
    194 
    195 Minimizing the corpus
    196 ---------------------
    197 
    198 When you have gathered corpus data from more than one fuzzer run
    199 or for any other reason want to minimize the data
    200 in some corpus subdirectory `fuzz/corpora/DIR` this can be done as follows:
    201 
    202     mkdir fuzz/corpora/NEWDIR
    203     fuzz/$FUZZER -merge=1 fuzz/corpora/NEWDIR fuzz/corpora/DIR
    204