Intelligent Systems
Note: This research group has relocated. Discover the updated page here

On Joint Estimation of Pose, Geometry and svBRDF from a Handheld Scanner

2020

Conference Paper

avg


We propose a novel formulation for joint recovery of camera pose, object geometry and spatially-varying BRDF. The input to our approach is a sequence of RGB-D images captured by a mobile, hand-held scanner that actively illuminates the scene with point light sources. Compared to previous works that jointly estimate geometry and materials from a hand-held scanner, we formulate this problem using a single objective function that can be minimized using off-the-shelf gradient-based solvers. By integrating material clustering as a differentiable operation into the optimization process, we avoid pre-processing heuristics and demonstrate that our model is able to determine the correct number of specular materials independently. We provide a study on the importance of each component in our formulation and on the requirements of the initial geometry. We show that optimizing over the poses is crucial for accurately recovering fine details and that our approach naturally results in a semantically meaningful material segmentation.

Author(s): Carolin Schmitt and Simon Donne and Gernot Riegler and Vladlen Koltun and Andreas Geiger
Book Title: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)
Year: 2020

Department(s): Autonomous Vision
Bibtex Type: Conference Paper (inproceedings)

Event Name: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Event Place: Seattle, USA

Links: pdf
Project Page
Slides
Video
Poster
Video:

BibTex

@inproceedings{Schmitt2020CVPR,
  title = {On Joint Estimation of Pose, Geometry and svBRDF from a Handheld Scanner},
  author = {Schmitt, Carolin and Donne, Simon and Riegler, Gernot and Koltun, Vladlen and Geiger, Andreas},
  booktitle = { Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
  year = {2020},
  doi = {}
}