Towards Extracting Faithful and Descriptive Representations of Latent Variable Models
Ivan Sanchez, Tim Rocktaschel, Sebastian Riedel, Sameer Singh

Citation
Ivan Sanchez, Tim Rocktaschel, Sebastian Riedel, Sameer Singh. "Towards Extracting Faithful and Descriptive Representations of Latent Variable Models". AAAI Spring Syposium on Knowledge Representation and Reasoning (KRR): Integrating Symbolic and Neural Approaches, March, 2015.

Abstract
Methods that use latent representations of data, such as matrix and tensor factorization or deep neural methods, are becoming increasingly popular for applications such as knowledge base population and recommendation systems. These approaches have been shown to be very robust and scalable but, in contrast to more symbolic approaches, lack interpretability. This makes debugging such models difficult, and might result in users not trusting the predictions of such systems. To overcome this issue we propose to extract an interpretable proxy model from a predictive latent variable model. We use a socalled pedagogical method, where we query our predictive model to obtain observations needed for learning a descriptive model. We describe two families of (presumably more) descriptive models, simple logic rules and Bayesian networks, and show how members of these families provide descriptive representations of matrix factorization models. Preliminary experiments on knowledge extraction from text indicate that even though Bayesian networks may be more faithful to a matrix factorization model than the logic rules, the latter are possibly more useful for interpretation and debugging.

Electronic downloads

Internal. This publication has been marked by the author for TerraSwarm-only distribution, so electronic downloads are not available without logging in.

Citation formats

• HTML

  Ivan Sanchez, Tim Rocktaschel, Sebastian Riedel, Sameer
  Singh. <a
  href="http://www.terraswarm.org/pubs/482.html"
  >Towards Extracting Faithful and Descriptive
  Representations of Latent Variable Models</a>, AAAI
  Spring Syposium on Knowledge Representation and Reasoning
  (KRR): Integrating Symbolic and Neural Approaches, March,
  2015.

• Plain text

  Ivan Sanchez, Tim Rocktaschel, Sebastian Riedel, Sameer
  Singh. "Towards Extracting Faithful and Descriptive
  Representations of Latent Variable Models". AAAI Spring
  Syposium on Knowledge Representation and Reasoning (KRR):
  Integrating Symbolic and Neural Approaches, March, 2015.

• BibTeX

  @inproceedings{SanchezRocktaschelRiedelSingh15_TowardsExtractingFaithfulDescriptiveRepresentationsOf,
      author = {Ivan Sanchez and Tim Rocktaschel and Sebastian
                Riedel and Sameer Singh},
      title = {Towards Extracting Faithful and Descriptive
                Representations of Latent Variable Models},
      booktitle = {AAAI Spring Syposium on Knowledge Representation
                and Reasoning (KRR): Integrating Symbolic and
                Neural Approaches},
      month = {March},
      year = {2015},
      abstract = {Methods that use latent representations of data,
                such as matrix and tensor factorization or deep
                neural methods, are becoming increasingly popular
                for applications such as knowledge base population
                and recommendation systems. These approaches have
                been shown to be very robust and scalable but, in
                contrast to more symbolic approaches, lack
                interpretability. This makes debugging such models
                difficult, and might result in users not trusting
                the predictions of such systems. To overcome this
                issue we propose to extract an interpretable proxy
                model from a predictive latent variable model. We
                use a socalled pedagogical method, where we query
                our predictive model to obtain observations needed
                for learning a descriptive model. We describe two
                families of (presumably more) descriptive models,
                simple logic rules and Bayesian networks, and show
                how members of these families provide descriptive
                representations of matrix factorization models.
                Preliminary experiments on knowledge extraction
                from text indicate that even though Bayesian
                networks may be more faithful to a matrix
                factorization model than the logic rules, the
                latter are possibly more useful for interpretation
                and debugging.},
      URL = {http://terraswarm.org/pubs/482.html}
  }
  

Posted by Sameer Singh on 31 Jan 2015.
Groups: services