Team for Research in
Ubiquitous Secure Technology

Citation
Marjan Aslani, NGA CHUNG, Jason Doherty, Nichole Stockman, William Quach. "Comparison of Blackbox and Whitebox Fuzzers in Finding Software Bugs". Talk or presentation, 12, November, 2008.

Abstract
Both blackbox and whitebox fuzzing techniques have been widely used to uncover security vulnerabilities in software applications, but there have been few studies comparing each technique. Our approach was to use Zzuf, a blackbox fuzzer, and Catchconv, a whitebox fuzzer, to generate test cases that were then run on open source and commercial software to compare both fuzzers efciency in terms of the number of unique bugs found per test case. An analysis of our results showed that Zzuf found an average of 2:69 unique errors per 100 unique test cases, while Catchconv found an average of 2:63 unique errors per 100 test cases. In terms of unique errors per total errors, 22 percent of the total errors found by Catchconv were unique, while 0:05 percent of the total errors found by Zzuf were unique. From the analysis of the data we collected, we identified metrics which we suggest for future comparison between fuzzers, but we did not collect enough information to evaluate in our study.

Electronic downloads

• 22%20-%20Aslani.ppt · application/vnd.ms-powerpoint · 999 kbytes

• HTML

  Marjan Aslani, NGA CHUNG, Jason Doherty, Nichole Stockman,
  William Quach. <a
  href="http://www.truststc.org/pubs/493.html"
  ><i>Comparison of Blackbox and Whitebox Fuzzers in
  Finding Software Bugs</i></a>, Talk or
  presentation,  12, November, 2008.

• Plain text

  Marjan Aslani, NGA CHUNG, Jason Doherty, Nichole Stockman,
  William Quach. "Comparison of Blackbox and Whitebox
  Fuzzers in Finding Software Bugs". Talk or
  presentation,  12, November, 2008.

• BibTeX

  @presentation{AslaniCHUNGDohertyStockmanQuach08_ComparisonOfBlackboxWhiteboxFuzzersInFindingSoftware,
      author = {Marjan Aslani and NGA CHUNG and Jason Doherty and
                Nichole Stockman and William Quach},
      title = {Comparison of Blackbox and Whitebox Fuzzers in
                Finding Software Bugs},
      day = {12},
      month = {November},
      year = {2008},
      abstract = {Both blackbox and whitebox fuzzing techniques have
                been widely used to uncover security
                vulnerabilities in software applications, but
                there have been few studies comparing each
                technique. Our approach was to use Zzuf, a
                blackbox fuzzer, and Catchconv, a whitebox fuzzer,
                to generate test cases that were then run on open
                source and commercial software to compare both
                fuzzers efciency in terms of the number of unique
                bugs found per test case. An analysis of our
                results showed that Zzuf found an average of 2:69
                unique errors per 100 unique test cases, while
                Catchconv found an average of 2:63 unique errors
                per 100 test cases. In terms of unique errors per
                total errors, 22 percent of the total errors found
                by Catchconv were unique, while 0:05 percent of
                the total errors found by Zzuf were unique. From
                the analysis of the data we collected, we
                identified metrics which we suggest for future
                comparison between fuzzers, but we did not collect
                enough information to evaluate in our study. },
      URL = {http://www.truststc.org/pubs/493.html}
  }
  

Posted by Jessica Gamble on 23 Jan 2009.
For additional information, see the Publications FAQ or contact webmaster at www truststc org.