Fuzz Testing Rundown
5 min read
Fuzzing is a software testing methodology that is great at finding obscure bugs that developers miss. Instead of testing with a small, pre-defined set of cases (like unit testing), fuzzing tests code 24 hours a day, using the feedback and results it gathers to generate new cases (called fuzz), in an effort to exercise all aspects of the software in question. You can think of fuzzing as a feedback loop that iterates thousands of times per second:
Fuzzing Feedback Loop
Fuzzing finds the hard-to-find bugs that other security tools (SAST, SCA, etc) typically miss.
Unit testing, and other developer-driven testing methods like static analysis are great for sanity checks, and ensuring that your code handles all of the cases your developers have thought about. Where unit tests fall short is in the unknown; they look for exactly what you ask them to, and nothing more.
By continuously fuzz testing your code, you immediately increase your bug-finding potential to include the unforeseen problems that aren't covered by static analysis rules or your developers' imaginations. Fuzzing covers an area of your software test pipeline that no other type of testing methodology touches.
With fuzzing you can find:
Types of Fuzzing
- Generates completely random test cases, or randomly mutates & corrupts example inputs
- Has no understanding of the code being tested, and can’t intelligently generate tests to drive deeper into a codebase
- Only finds very superficial bugs, because the fuzzer’s shallow coverage won’t enable any testing below the parsing layer
- Generates tests that conform to a specific protocol, creating inputs that are structurally correct, but provide invalid data (HTTP, Bluetooth, HL7, JSON, XML, etc)
- Does not require source code modifications, simply select the protocol that your application uses
- The ability to generate structurally-valid inputs allows protocol fuzzing to test the logic of your application and uncover deeper bugs
Coverage Guided Fuzzing
- Uses code coverage feedback to guide test case generation, allowing the fuzzer to build off previously generated inputs
- Code coverage enables the fuzzer to work out which sections of a test case affect certain areas of the code, effectively drilling down into high risk areas
- Discovers hard-to-find (& easily missed) bugs on rare code-paths by ensuring that every line of code is covered