Main navigation | Main content
In this week's lab you'll try out the fuzzing tool AFL, to find interesting crashing inputs of programs. (To be precise we'll be using AFL++, a more recent forked version.)
As usual in the online lab we'll randomly split you into breakout groups of 2-3 students: please work together, discuss, and learn from the other student(s) in you group. Use the "Ask for Help" button to ask questions or show off what you've done. We also recommend working in groups in the in-person lab, but there you can choose your own groups and physically raise your hand to ask a question. You may still find it useful to use Zoom or tmate for screen sharing in person while respecting social distancing.
We'll walk through using AFL on two different example programs, a very contrived example based on a text adventure game, and the slightly more realistic example of the bcimgview program from project 1.
A good starting point for the documentation of AFL++ is a long README file that you can find on the AFL++ Github front page. It's too long to suggest you read through it all during lab, though. Another interesting documentation sub-page is the explanation of what all the console statistics mean. It's also pretty long, but you might skim though it if you're otherwise just watching the statistics screen waiting for something interesting to happen.
Because fuzzing involves creating, using, and removing files quickly, it will work noticeably faster if the files are kept on a local filesystem rather than a networked one like the CSE Labs home directories. Also we discovered that the version of AFL we compiled for this lab doesn't work on Vole. So if you are using the Vole GUI environment, you should still SSH from there to a more recent CSE Labs machine like the 1-250 or 4-250 lab machines. And then on that machine, we suggest creating yourself a directory under /export/scratch/users and doing your work there. The convention would be to name the subdirectory of users after your username, as in:
mkdir /export/scratch/users/goldy007 cd /export/scratch/users/goldy007 mkdir 4271-afl-lab cd 4271-afl-lab
There are three programs from AFL++ that you'll need to use. afl-cc is a compiler that adds control flow instrumentation to make a binary suitable for use with AFL/AFL++, and afl-fuzz is the fuzzer itself. afl-tmin is a program to automatically simplify test cases. Since the location of these programs in the course directory is long, we suggest making symlinks to them in your current directory, as in the following commands.
ln -s /web/classes/Fall-2020/csci4271/soft/afl/bin/{afl-cc,afl-fuzz,afl-tmin} .
(Or you could also add the bin directory to your path.)
Our first example is a modeled after a text adventure game, where getting the program to crash is like a game event. You can try compiling the program normally and running it with commands on the standard input. It's simple enough that with a little bit of experimenting and/or reading the source code you should be able to find how to get to the magic potion.
cp /web/classes/Fall-2020/csci4271/labs/10/maze.c . gcc -Wall -g maze.c -o maze ./maze
Next let's see if AFL can find the potion (crash) as well. First recompile the program using afl-cc:
./afl-cc -g maze.c -o maze-afl
Though the maze that AFL needs to explore here isn't really that large, changing the input to the game randomly would take a long time to get it to the goal, because there are only a few legal commands. So the thing we can do that is most useful is to give it a dictionary of the legal commands in the game. This is like a simplified form of grammar-based fuzzing, where we just provide some useful tokens rather than a full grammar. We've supplied a sample dictionary you can use:
cp /web/classes/Fall-2020/csci4271/labs/10/maze.dict .
The other thing we have to give to get AFL started is a directory of seed inputs. A bunch of good seeds are potentially another way to give AFL information about what the input format should look like. On the other hand large seeds can cause some parts of AFL to slow down, so you can potentially put a lot of work into optimal seeds. But because we're already helping with the dictionary, choosing good inputs turns out not to be as important. Let's create a minimal set with just one:
mkdir maze-inputs echo 'go north' >maze-inputs/input1
One other piece of trivia to deal with is that AFL suggests setting up a system option to make it run faster, but you won't be allowed to change that option on CSE Labs, so we need to use an environment variable to tell AFL not to worry. So putting together all those resources, the command to start AFL looks like:
env AFL_SKIP_CPUFREQ=1 ./afl-fuzz -i maze-inputs -x maze.dict -o maze-results -- ./maze-afl
Once you start AFL running, it will take over its terminal with a bunch of statistics about the execution. While it is running, it will fill the directory specified with -o, maze-results in our example command, with the interesting inputs it finds: inputs that cause new code to be executed, inputs that cause crashes, and inputs that cause execution to take much longer than expected (called hangs). These input files are kept in subdirectories of maze-results, specifically ones named default/queue, default/crashes, and default/hangs. If you're waiting to get good results as shown by the execution statistics, you can open another terminal at the same time and use it to look at the generated files. The files in the queue directory will give you an idea of how AFL is exploring the search space, while the crashes and hangs are the results that it has found so far. The file names in these directories have long names that give some information about what part of fuzzing process produced them.
The maze example should run pretty quickly. If you're using a scratch drive on a lab machine, you should see it able to execute around 6000 tests per second (shown as exec speed) in the statistics, and it should start finding crashes within a minute or two.
Because the maze program is tolerant of a lot of junk in its input (unknown commands are just ignored), AFL's default mode will produce long crashing inputs with a mix of legal commands and random data. If you just wanted to confirm that the program had a bug or trigger it under the debugger, this would be enough. But for understanding the program it would be nice to have cleaner-looking crashing inputs. AFL has a companion tool based on the same execution experiment infrastructure that searches for ways to make test inputs smaller, which generally also makes them cleaner-looking. You can run it on one of the crashes you've found using a command like:
./afl-tmin -i maze-results/default/crashes/id:000000* -o crash-reduced.in -- ./maze-afl
That particular command will try to minimize the first crash, or you can replace the id:000000* with the name of a particular test case you're interested in. The output file crash-reduced.in will be a simplified crashing input.
Another example of a C program you might be interested in crashing bugs in is bcimgview, the buggy image-parsing program from Project 1. We've also prepared a version of it you can test using a similar process as with the maze program. To make the compilation simpler we've made a version without the GUI:
cp /web/classes/Fall-2020/csci4271/labs/10/bcimgview-no-gui.c . ./afl-cc -g bcimgview-no-gui.c -o bcimgview-afl
For binary formats like bcimgview uses, good seed inputs can be more valuable than a dictionary. A small starting point would be to use the smallest images from the sample images:
cp /web/classes/Fall-2020/csci4271/labs/10/flag.bc* images-small
Because this program still needs the -c flag, one thing that should be slightly different in the command is how to tell AFL to run the binary: you give it a template that can contain other options, but then has the location where the input filename should go replaced with @@. So for instance the command might look like:
env AFL_SKIP_CPUFREQ=1 ./afl-fuzz -i images-small -o results2 -- ./bcimgview-afl -c @@
With the default settings you'll seem some inputs that AFL labels as crashes almost immediately, but the ones you'll see first are disappointing from a security vulnerability standpoint because they are just assertion failures. You can tell this because their filenames have sig:06; signal number 6 on Unix is the abort signal used by an assertion failure. But if you let it run longer you should see some crashing inputs whose names include sig:11, which are real segmentation faults. You could also try disabling the assertions in the source code if you'd like. Part of the AFL-oriented compilation we did with these programs also helps them print a little more information when they segfault.
The existence of these crashing inputs proves that something is going wrong inside bcimgview, but because they are just binary files, they don't immediately tell you what the bug is, or whether it is exploitable. (This is part of why we didn't recommend you use a tool like AFL for the first part of project 1.) Some steps you could take to investigate what's going wrong might include running the tests under Valgrind, running them under GDB, and comparing values you see in the crash report message or in GDB to values in the input.
As an exception to the normal collaboration rules about Project 1, you can talk in with your labmates about how the crashing bcimgview inputs you found with AFL relate to the program source code and bugs. But limit this to what you can find from the AFL test cases, not what you found from your other manual investigations.