Fuzzing
Fuzzing feeds a program huge numbers of generated or mutated inputs, many of them malformed, and watches for crashes, hangs, memory errors and wrong results. Coverage-guided engines such as libFuzzer keep the inputs that reach new code, so they work deeper into a format over time.
Parsers break on inputs nobody writes a test for: a length field that promises more bytes than arrive, a truncated header, an archive inside an archive. A fuzzer tries millions of them. CVE-2022-37434, a heap over-read or overflow in zlib's inflate on a large gzip header extra field, is the kind of bug it reaches quickly, and in C and C++ sanitizers turn silent memory corruption into a crash it can report.
- Who
- The developers who own the parser write the harness; AppSec or a platform team runs the fuzzing infrastructure
- When
- When code starts parsing untrusted files, archives, messages or other complex input, and again whenever that code changes
- How often
- A short run on each pull request that touches the parser, and longer runs nightly or continuously
- You end up with
- A harness per parser, a stored corpus, a coverage report, and every minimized crash input kept as a regression test
Do this
List the code that parses untrusted input
Start with code that reads bytes from outside: file and image formats, archives, network protocols, serialization and query languages. A common order is native C or C++ parsers of uploaded or downloaded data first, then memory-safe code, where panics, hangs and huge allocations still cause outages. OWASP SAMM asks for fuzzing of at least the application's main input parameters.
Write a narrow, deterministic harness
Write one fuzz target per entry point that passes the fuzzer's bytes to the real parser:
LLVMFuzzerTestOneInputin C and C++, orfunc FuzzParse(f *testing.F)in Go. libFuzzer's documentation asks for targets that are deterministic, fast and narrow, and that avoid changing global state. Add assertions so an accepted input must mean what it claims, not only avoid crashing.Build native code with sanitizers
Compile the harness and parser with AddressSanitizer and UndefinedBehaviorSanitizer, for example
clang -g -O1 -fsanitize=fuzzer,address,undefined. Without them an out-of-bounds read may never crash, so the fuzzer never reports it. AddressSanitizer catches out-of-bounds access, use-after-free and double free, at a typical slowdown of about 2x.Seed a corpus and use your language's engine
Give the fuzzer small valid samples and a few truncated ones so it gets past the first header check, plus a dictionary of format keywords (
-dictin libFuzzer). Use the engine your language provides; the table below lists the common ones.Run it on pull requests and on a schedule
Run a time-boxed job on pull requests that touch the parser, such as
go test -fuzz=FuzzParse -fuzztime=60sor libFuzzer with-max_total_time=60, and longer runs nightly. ClusterFuzzLite runs both modes in GitHub Actions, GitLab and other CI systems; OSS-Fuzz is a free service that fuzzes open-source projects continuously. Keep the corpus between runs.Minimize every crash and keep it as a test
Reproduce the crash from the saved input (libFuzzer writes
crash-<sha1>), shrink it with-minimize_crash=1, and fix the rule the input broke rather than the one value. Add the minimized input to the unit tests. Go does this for you: it writes the failing input undertestdata/fuzz/, and plaingo testreplays it from then on.Check what the fuzzer actually reaches
Read the coverage report from ClusterFuzzLite or your engine and confirm the fuzzer reaches the payload handling, not only the first header check. Once, run the harness against an old version with a known bug: if it does not find the bug, the harness is too shallow.
Done when
Fuzzing runs on a schedule, and each crash it finds becomes a test.
Mark it in the gap checkCommon mistakes
- Fuzzing native code without sanitizers. An out-of-bounds read that does not crash goes unreported, and the run looks clean.
- Patching the crashing value, such as rejecting a length of 8, instead of the check it broke. The next truncated input fails the same way.
- Deleting crash files once the bug is fixed. Without the minimized input in the test suite, a later refactor can bring the bug back unnoticed.
- A harness that rejects nearly every input at the first magic-byte check. The fuzzer burns CPU at the door; seed it with valid samples and check coverage.
- Reading execution counts as proof. Millions of inputs describe the experiment, not the absence of bugs, and a harness that resets state each time cannot find a bug that needs an earlier message to set a mode.
Which fuzzing engine for which language?
libFuzzer now receives bug fixes only, but a harness written to its LLVMFuzzerTestOneInput entry point also runs under AFL++, honggfuzz, OSS-Fuzz and ClusterFuzzLite.
| Language | Engine | Run continuously with |
|---|---|---|
| C and C++ | libFuzzer, AFL++, honggfuzz | OSS-Fuzz or ClusterFuzzLite |
| Go | go test -fuzz (Go 1.18 and later) | A time-boxed go test -fuzz job, or ClusterFuzzLite |
| Rust | cargo-fuzz (libFuzzer) | OSS-Fuzz or ClusterFuzzLite |
| Python | Atheris (based on libFuzzer) | OSS-Fuzz or ClusterFuzzLite |
| Java and other JVM languages | Jazzer (based on libFuzzer) | OSS-Fuzz or ClusterFuzzLite |
Tools and guides
- Workflow: Test an application
- Fuzz a parser and turn crashes into lasting regressions
Build a narrow harness, observe memory and resource failures, minimize the input, and test the repaired parser.
- Find, fix and contain memory-safety bugs in C and C++
Recognize out-of-bounds access, use after free, NULL dereference and integer overflow in C and C++, fix them with checked lengths and single ownership, harden release builds, catch the rest with sanitizers and fuzzing, and plan new code in a memory-safe language.
- Place each security control in the pipeline and decide what it blocks
Put secrets scanning, SAST, SCA, IaC and image scanning, DAST, fuzzing, attestation and admission control where each sees the right input, block only on findings a developer can act on, and deploy only images the pipeline has attested.
Standards
- NIST SSDF 1.1
- PW.8 Test Executable Code to Identify Vulnerabilities and Verify Compliance with Security Requirements
- OWASP SAMM 2
- Requirements-driven TestingSecurity Testing
Sources
- LLVM: libFuzzer, a library for coverage-guided fuzz testing
- Clang: AddressSanitizer
- Go: Fuzzing
- ClusterFuzzLite documentation
- OSS-Fuzz documentation
- Atheris: a coverage-guided Python fuzzing engine
- Jazzer: coverage-guided fuzzing for the JVM
- cargo-fuzz: fuzzing Rust with libFuzzer
- OWASP: Fuzzing
- OWASP SAMM: Requirements-driven Testing, Misuse/Abuse Testing stream
- NVD: CVE-2022-37434, heap overflow in zlib inflate
- NIST SP 800-218: Secure Software Development Framework (SSDF) Version 1.1. February 2022.