Real-Time Non-Rigid Multi-Frame Depth Video Super-Resolution

Kassem Al Ismaeil, Djamila Aouada, Thomas Solignac, Bruno Mirbach, Bjorn Ottersten; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2015, pp. 8-16

Abstract


This paper proposes to enhance low resolution dynamic depth videos containing freely non-rigidly moving objects with a new dynamic multi-frame super-resolution algorithm. Existent methods are either limited to rigid objects, or restricted to global lateral motions discarding radial displacements. We address these shortcomings by accounting for non-rigid displacements in 3D. In addition to 2D optical flow, we estimate the depth displacement, and simultaneously correct the depth measurement by Kalman filtering. This concept is incorporated efficiently in a multi-frame super-resolution framework. This is formulated in a recursive manner that ensures an efficient deployment in real-time. Results show the overall improved performance of the proposed method as compared to alternative approaches, specifically in handling relatively large 3D motions. Test examples range from a full moving human body to a highly dynamic facial video with varying expressions.

Related Material


[pdf]
[bibtex]
@InProceedings{Ismaeil_2015_CVPR_Workshops,
author = {Al Ismaeil, Kassem and Aouada, Djamila and Solignac, Thomas and Mirbach, Bruno and Ottersten, Bjorn},
title = {Real-Time Non-Rigid Multi-Frame Depth Video Super-Resolution},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2015}
}