Software Securityhome
Fix a findingSearch
Phase 4 · Testing

Fuzzing

Fuzzing feeds a program huge numbers of generated or mutated inputs, many of them malformed, and watches for crashes, hangs, memory errors and wrong results. Coverage-guided engines such as libFuzzer keep the inputs that reach new code, so they work deeper into a format over time.

Parsers break on inputs nobody writes a test for: a length field that promises more bytes than arrive, a truncated header, an archive inside an archive. A fuzzer tries millions of them. CVE-2022-37434, a heap over-read or overflow in zlib's inflate on a large gzip header extra field, is the kind of bug it reaches quickly, and in C and C++ sanitizers turn silent memory corruption into a crash it can report.

Who
The developers who own the parser write the harness; AppSec or a platform team runs the fuzzing infrastructure
When
When code starts parsing untrusted files, archives, messages or other complex input, and again whenever that code changes
How often
A short run on each pull request that touches the parser, and longer runs nightly or continuously
You end up with
A harness per parser, a stored corpus, a coverage report, and every minimized crash input kept as a regression test

Do this

  1. List the code that parses untrusted input

    Start with code that reads bytes from outside: file and image formats, archives, network protocols, serialization and query languages. A common order is native C or C++ parsers of uploaded or downloaded data first, then memory-safe code, where panics, hangs and huge allocations still cause outages. OWASP SAMM asks for fuzzing of at least the application's main input parameters.

  2. Write a narrow, deterministic harness

    Write one fuzz target per entry point that passes the fuzzer's bytes to the real parser: LLVMFuzzerTestOneInput in C and C++, or func FuzzParse(f *testing.F) in Go. libFuzzer's documentation asks for targets that are deterministic, fast and narrow, and that avoid changing global state. Add assertions so an accepted input must mean what it claims, not only avoid crashing.

  3. Build native code with sanitizers

    Compile the harness and parser with AddressSanitizer and UndefinedBehaviorSanitizer, for example clang -g -O1 -fsanitize=fuzzer,address,undefined. Without them an out-of-bounds read may never crash, so the fuzzer never reports it. AddressSanitizer catches out-of-bounds access, use-after-free and double free, at a typical slowdown of about 2x.

  4. Seed a corpus and use your language's engine

    Give the fuzzer small valid samples and a few truncated ones so it gets past the first header check, plus a dictionary of format keywords (-dict in libFuzzer). Use the engine your language provides; the table below lists the common ones.

  5. Run it on pull requests and on a schedule

    Run a time-boxed job on pull requests that touch the parser, such as go test -fuzz=FuzzParse -fuzztime=60s or libFuzzer with -max_total_time=60, and longer runs nightly. ClusterFuzzLite runs both modes in GitHub Actions, GitLab and other CI systems; OSS-Fuzz is a free service that fuzzes open-source projects continuously. Keep the corpus between runs.

  6. Minimize every crash and keep it as a test

    Reproduce the crash from the saved input (libFuzzer writes crash-<sha1>), shrink it with -minimize_crash=1, and fix the rule the input broke rather than the one value. Add the minimized input to the unit tests. Go does this for you: it writes the failing input under testdata/fuzz/, and plain go test replays it from then on.

  7. Check what the fuzzer actually reaches

    Read the coverage report from ClusterFuzzLite or your engine and confirm the fuzzer reaches the payload handling, not only the first header check. Once, run the harness against an old version with a known bug: if it does not find the bug, the harness is too shallow.

Done when

Fuzzing runs on a schedule, and each crash it finds becomes a test.

Mark it in the gap check

Common mistakes

  • Fuzzing native code without sanitizers. An out-of-bounds read that does not crash goes unreported, and the run looks clean.
  • Patching the crashing value, such as rejecting a length of 8, instead of the check it broke. The next truncated input fails the same way.
  • Deleting crash files once the bug is fixed. Without the minimized input in the test suite, a later refactor can bring the bug back unnoticed.
  • A harness that rejects nearly every input at the first magic-byte check. The fuzzer burns CPU at the door; seed it with valid samples and check coverage.
  • Reading execution counts as proof. Millions of inputs describe the experiment, not the absence of bugs, and a harness that resets state each time cannot find a bug that needs an earlier message to set a mode.

Which fuzzing engine for which language?

libFuzzer now receives bug fixes only, but a harness written to its LLVMFuzzerTestOneInput entry point also runs under AFL++, honggfuzz, OSS-Fuzz and ClusterFuzzLite.

Which fuzzing engine for which language?: table
LanguageEngineRun continuously with
C and C++libFuzzer, AFL++, honggfuzzOSS-Fuzz or ClusterFuzzLite
Gogo test -fuzz (Go 1.18 and later)A time-boxed go test -fuzz job, or ClusterFuzzLite
Rustcargo-fuzz (libFuzzer)OSS-Fuzz or ClusterFuzzLite
PythonAtheris (based on libFuzzer)OSS-Fuzz or ClusterFuzzLite
Java and other JVM languagesJazzer (based on libFuzzer)OSS-Fuzz or ClusterFuzzLite

Tools and guides

Standards

NIST SSDF 1.1
PW.8 Test Executable Code to Identify Vulnerabilities and Verify Compliance with Security Requirements

Sources

  1. LLVM: libFuzzer, a library for coverage-guided fuzz testing
  2. Clang: AddressSanitizer
  3. Go: Fuzzing
  4. ClusterFuzzLite documentation
  5. OSS-Fuzz documentation
  6. Atheris: a coverage-guided Python fuzzing engine
  7. Jazzer: coverage-guided fuzzing for the JVM
  8. cargo-fuzz: fuzzing Rust with libFuzzer
  9. OWASP: Fuzzing
  10. OWASP SAMM: Requirements-driven Testing, Misuse/Abuse Testing stream
  11. NVD: CVE-2022-37434, heap overflow in zlib inflate
  12. NIST SP 800-218: Secure Software Development Framework (SSDF) Version 1.1. February 2022.

Find your next answer

Type to search. Search runs in your browser.

Searches run in your browser, but the host can infer them from the index files they load. The address shows your search so you can bookmark or share it.