3D All The Way: Semantic Segmentation of Urban Scenes From Start to End in 3D

Andelo Martinovic, Jan Knopp, Hayko Riemenschneider, Luc Van Gool; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 4456-4465

Abstract


We propose a new approach for semantic segmentation of 3D city models. Starting from an SfM reconstruction of a street-side scene, we perform classification and facade splitting purely in 3D, obviating the need for slow image-based semantic segmentation methods. We show that a properly trained pure-3D approach produces high quality labelings, with significant speed benefits (20x faster) allowing us to analyze entire streets in a matter of minutes. Additionally, if speed is not of the essence, the 3D labeling can be combined with the results of a state-of-the-art 2D classifier, further boosting the performance. Further, we propose a novel facade separation based on semantic nuances between facades. Finally, inspired by the use of architectural principles for 2D facade labeling, we propose new 3D-specific principles and an efficient optimization scheme based on an integer quadratic programming formulation.

Related Material


[pdf]
[bibtex]
@InProceedings{Martinovic_2015_CVPR,
author = {Martinovic, Andelo and Knopp, Jan and Riemenschneider, Hayko and Van Gool, Luc},
title = {3D All The Way: Semantic Segmentation of Urban Scenes From Start to End in 3D},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2015}
}