FrameBreak: Dramatic Image Extrapolation by Guided Shift-Maps

Yinda Zhang, Jianxiong Xiao, James Hays, Ping Tan; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 1171-1178


We significantly extrapolate the field of view of a photograph by learning from a roughly aligned, wide-angle guide image of the same scene category. Our method can extrapolate typical photos into complete panoramas. The extrapolation problem is formulated in the shift-map image synthesis framework. We analyze the self-similarity of the guide image to generate a set of allowable local transformations and apply them to the input image. Our guided shift-map method preserves to the scene layout of the guide image when extrapolating a photograph. While conventional shiftmap methods only support translations, this is not expressive enough to characterize the self-similarity of complex scenes. Therefore we additionally allow image transformations of rotation, scaling and reflection. To handle this increase in complexity, we introduce a hierarchical graph optimization method to choose the optimal transformation at each output pixel. We demonstrate our approach on a variety of indoor, outdoor, natural, and man-made scenes.

Related Material

author = {Zhang, Yinda and Xiao, Jianxiong and Hays, James and Tan, Ping},
title = {FrameBreak: Dramatic Image Extrapolation by Guided Shift-Maps},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2013}