Citation
Marta Kwiatkowska. "Safety Verification of Deep Neural Networks". Talk or presentation, 8, November, 2016.

Abstract
Deep neural networks have achieved impressive experimental results in image classification, but can surprisingly be unstable with respect to adversarial perturbations, that is, minimal changes to the input image that cause the network to misclassify it. With potential applications including perception modules and end-to-end controllers for self-driving cars, this raises concerns about their safety. We develop the first SMT-based automated verification framework for feed-forward multi-layer neural networks that works directly with the code of the network, exploring it layer by layer. We define safety for a region around a data point in a given layer by requiring that all points in the region are assigned the same class label. Working with a notion of a manipulation, a mapping between points that intuitively corresponds to a modification of an image, we employ discretisation to enable exhaustive search of the region. Our method can guarantee that adversarial examples are found for the given region and set of manipulations. If found, adversarial examples can be shown to human testers and/or used to fine-tune the network, and otherwise the network is declared safe for the given parameters. We implement the techniques using Z3 and evaluate them on state-of-the-art networks, including regularised and deep learning networks.

Electronic downloads

• seminar.2016-11-08.Marta_Kwitskowska.pdf · application/pdf · 2721 kbytes

• HTML

  Marta Kwiatkowska. <a
  href="http://chess.eecs.berkeley.edu/pubs/1184.html"
  ><i>Safety Verification of Deep Neural
  Networks</i></a>, Talk or presentation,  8,
  November, 2016.

• Plain text

  Marta Kwiatkowska. "Safety Verification of Deep Neural
  Networks". Talk or presentation,  8, November, 2016.

• BibTeX

  @presentation{Kwiatkowska16_SafetyVerificationOfDeepNeuralNetworks,
      author = {Marta Kwiatkowska},
      title = {Safety Verification of Deep Neural Networks},
      day = {8},
      month = {November},
      year = {2016},
      abstract = {Deep neural networks have achieved impressive
                experimental results in image classification, but
                can surprisingly be unstable with respect to
                adversarial perturbations, that is, minimal
                changes to the input image that cause the network
                to misclassify it. With potential applications
                including perception modules and end-to-end
                controllers for self-driving cars, this raises
                concerns about their safety. We develop the first
                SMT-based automated verification framework for
                feed-forward multi-layer neural networks that
                works directly with the code of the network,
                exploring it layer by layer. We define safety for
                a region around a data point in a given layer by
                requiring that all points in the region are
                assigned the same class label. Working with a
                notion of a manipulation, a mapping between points
                that intuitively corresponds to a modification of
                an image, we employ discretisation to enable
                exhaustive search of the region. Our method can
                guarantee that adversarial examples are found for
                the given region and set of manipulations. If
                found, adversarial examples can be shown to human
                testers and/or used to fine-tune the network, and
                otherwise the network is declared safe for the
                given parameters. We implement the techniques
                using Z3 and evaluate them on state-of-the-art
                networks, including regularised and deep learning
                networks. },
      URL = {http://chess.eecs.berkeley.edu/pubs/1184.html}
  }
  

Posted by Sadigh Dorsa on 14 Nov 2016.
For additional information, see the Publications FAQ or contact webmaster at chess eecs berkeley edu.