Localization from Semantic Observations via the Matrix Permanent
Nikolay A. Atanasov, Menglong Zhu, Kostas Daniilidis, George Pappas

Citation
Nikolay A. Atanasov, Menglong Zhu, Kostas Daniilidis, George Pappas. "Localization from Semantic Observations via the Matrix Permanent". The International Journal of Robotics Research, 2015.

Abstract
Most approaches to robot localization rely on low-level geometric features such as points, lines, and planes. In this paper, we use object recognition to obtain semantic information from the robot's sensors and consider the task of localizing the robot within a prior map of landmarks, which are annotated with semantic labels. As object recognition algorithms miss detections and produce false alarms, correct data association between the detections and the landmarks on the map is central to the semantic localization problem. Instead of the traditional vector-based representation, we propose a sensor model, which encodes the semantic observations via random finite sets and enables a unified treatment of missed detections, false alarms, and data association. Our second contribution is to reduce the problem of computing the likelihood of a set-valued observation to the problem of computing a matrix permanent. It is this crucial transformation that allows us to solve the semantic localization problem with a polynomial-time approximation to the set-based Bayes filter. The performance of our approach is demonstrated in simulation and in a real environment using a deformable-part-model-based object detector. Robust global localization from semantic observations is demonstrated for a mobile robot and for the Project Tango phone. Comparisons are made with the traditional lidar-based geometric Monte-Carlo localization.

Electronic downloads

Internal. This publication has been marked by the author for TerraSwarm-only distribution, so electronic downloads are not available without logging in.

Citation formats

• HTML

  Nikolay A. Atanasov, Menglong Zhu, Kostas Daniilidis, George
  Pappas. <a
  href="http://www.terraswarm.org/pubs/458.html"
  >Localization from Semantic Observations via the Matrix
  Permanent</a>, <i>The International Journal of
  Robotics Research</i>,  2015.

• Plain text

  Nikolay A. Atanasov, Menglong Zhu, Kostas Daniilidis, George
  Pappas. "Localization from Semantic Observations via
  the Matrix Permanent". <i>The International
  Journal of Robotics Research</i>,  2015.

• BibTeX

  @article{AtanasovZhuDaniilidisPappas15_LocalizationFromSemanticObservationsViaMatrixPermanent,
      author = {Nikolay A. Atanasov and Menglong Zhu and Kostas
                Daniilidis and George Pappas},
      title = {Localization from Semantic Observations via the
                Matrix Permanent},
      journal = {The International Journal of Robotics Research},
      year = {2015},
      abstract = {Most approaches to robot localization rely on
                low-level geometric features such as points,
                lines, and planes. In this paper, we use object
                recognition to obtain semantic information from
                the robot's sensors and consider the task of
                localizing the robot within a prior map of
                landmarks, which are annotated with semantic
                labels. As object recognition algorithms miss
                detections and produce false alarms, correct data
                association between the detections and the
                landmarks on the map is central to the semantic
                localization problem. Instead of the traditional
                vector-based representation, we propose a sensor
                model, which encodes the semantic observations via
                random finite sets and enables a unified treatment
                of missed detections, false alarms, and data
                association. Our second contribution is to reduce
                the problem of computing the likelihood of a
                set-valued observation to the problem of computing
                a matrix permanent. It is this crucial
                transformation that allows us to solve the
                semantic localization problem with a
                polynomial-time approximation to the set-based
                Bayes filter. The performance of our approach is
                demonstrated in simulation and in a real
                environment using a deformable-part-model-based
                object detector. Robust global localization from
                semantic observations is demonstrated for a mobile
                robot and for the Project Tango phone. Comparisons
                are made with the traditional lidar-based
                geometric Monte-Carlo localization.},
      URL = {http://terraswarm.org/pubs/458.html}
  }
  

Posted by Nikolay A. Atanasov on 11 Nov 2014.
Groups: services