Render for CNN: Viewpoint Estimation in Images Using CNNs Trained With Rendered 3D Model Views

Hao Su, Charles R. Qi, Yangyan Li, Leonidas J. Guibas; The IEEE International Conference on Computer Vision (ICCV), 2015, pp. 2686-2694

Abstract


Object viewpoint estimation from 2D images is an essential task in computer vision. However, two issues hinder its progress: scarcity of training data with viewpoint annotations, and a lack of powerful features. Inspired by the growing availability of 3D models, we propose a framework to address both issues by combining render-based image synthesis and CNNs (Convolutional Neural Networks). We believe that 3D models have the potential in generating a large number of images of high variation, which can be well exploited by deep CNN with a high learning capacity. Towards this goal, we propose a scalable and overfit-resistant image synthesis pipeline, together with a novel CNN specifically tailored for the viewpoint estimation task. Experimentally, we show that the viewpoint estimation from our pipeline can significantly outperform state-of-the-art methods on PASCAL 3D+ benchmark.

Related Material


[pdf]
[bibtex]
@InProceedings{Su_2015_ICCV,
author = {Su, Hao and Qi, Charles R. and Li, Yangyan and Guibas, Leonidas J.},
title = {Render for CNN: Viewpoint Estimation in Images Using CNNs Trained With Rendered 3D Model Views},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {December},
year = {2015}
}