Home

JQF is a feedback-directed fuzz testing platform for Java.

JQF is built on top of junit-quickcheck, which itself lets you write Quickcheck-like generators and properties in a Junit-style test class. JQF enables better input generation using state-of-the-art fuzzing tools such as AFL.

Quickstart

Write a Junit-like test class annotated with @RunWith(JQF.class) and write some test methods annotated with @Fuzz. The arguments to the test methods will be fuzzed by JQF:

@RunWith(JQF.class)
public class DateFormatterTest {

    /* Input params will be generated by JQF, many times. */
    /* Exceptions listed in the "throws" clause are considered normal (tests will pass on throw) */
    @Fuzz
    public void fuzzSimple(Date date, String format) throws IllegalArgumentException {
        // Create a simple date formatter using the input format string
        // May throw IllegalArgumentException for invalid formats
        DateFormat df = new SimpleDateFormat(format);

        // Format the date using the constructed formatter
        df.format(date);
    }
}

Fuzz with AFL:

$ jqf/bin/jqf-afl-fuzz -o results_dir DateFormatterTest fuzzSimple

Grab a coffee while AFL does its thing:

TODO: Paste image of AFL finding a crash

Ooh! We found a crash. Let's reproduce the test case and see what the error was.

$ jqf/bin/jqf-repro DateFormatterTest fuzzSimple results_dir/crashes/id:000000
java.lang.ArrayIndexOutOfBoundsException: 127
	at java.text.SimpleDateFormat.subFormat(SimpleDateFormat.java)
	at java.text.SimpleDateFormat.format(SimpleDateFormat.java)
	at java.text.SimpleDateFormat.format(SimpleDateFormat.java)
	at java.text.DateFormat.format(DateFormat.java)

This shouldn't happen! DateFormat.format() does not specify that it will throw ArrayIndexOutOfBoundsException. Time to file a bug report :-)

How does this work?

Unlike vanilla Quickcheck, which generates inputs using a random number generator, JQF receives feedback about the test program (such as the execution path for a given test case) that can be used to better guide input generation and reach corner cases that would not be executed by random sampling alone. Thus, JQF enables coverage-guided mutational fuzzing, among other things.

JQF receives feedback from the program by performing on-the-fly bytecode instrumentation using the ASM toolkit. JQF is quite fast, and can usually execute several hundred test cases per second for complex tests such as HTTP request parsing using Apache Tomcat and PNG decoding using Java ImageIO.

More details

Building JQF

Getting started with JQF is easy. It mostly uses Apache Maven to manage its build infrastructure. The README typically has the most up-to-date instructions on requirements as well as instructions to compile and run.

Using JQF

Writing tests

JQF builds on top of junit-quickcheck, so any valid quickcheck property and input generator works with JQF. The junit-quickcheck documentation is a great place to learn to write quickcheck tests and generators. Just remember to update the annotations on your test class from @RunWith(JUnitQuickcheck.class) to @RunWith(JQF.class) and the annotations on your test methods from @Property to @Fuzz if you want them to work with JQF.

For a JQF-centric tutorial on writing an effective fuzz target, see Writing a JQF test.

Fuzzing with AFL

JQF ships with out-of-the-box support for fuzzing with American Fuzzy Lop (AFL). AFL has been used to find numerous bugs in C (and LLVM) programs using lightweight instrumentation and coverage-guided fuzzing. If you've never used AFL before, the AFL quickstart guide may be a worthwhile read. In a nutshell, you want to run afl-fuzz on an instrumented program (known as the target), providing an input and output directory that respectively contain the input files to start with (known as the seeds) and where AFL will save the fuzzed inputs that it considers interesting (either because they found new code coverage or they resulted in a crash). JQF lets you run AFL on Java programs, so you can skip the steps relating to instrumenting C programs with afl-gcc or afl-clang. Instead, you give AFL the target program jqf-afl, which takes as arguments the Java test class and method that you want to fuzz. Simple enough. See Fuzzing Java with AFL for more details.

Hacking on JQF

One of JQF's core features is its ease of extensibility. Although a coverage-guided fuzz-with-AFL mode is provided out of the box, you can easily change the front-end to use different types of feedback (other than branch coverage) and input generation techniques other than AFL. See Extending JQF: The Guidance interface.

If you delve into the API exposed by JQF, you may also be interested in knowing more about the program instrumentation that JQF uses and the way it generates trace events for use in guided fuzzing. We've outlined some key points in the article on Implementation details.

Contact the developers

We want your feedback! (haha, get it? get it?)

If you've found a bug or are having trouble getting JQF to work, please open an issue on the issue tracker. You can also use this platform to post feature requests.

If it's some sort of fuzzing emergency you can always send an email to the main developer: Rohan Padhye.