Efficient ConvNet-Based Marker-Less Motion Capture in General Scenes With a Low Number of Cameras

Ahmed Elhayek, Edilson de Aguiar, Arjun Jain, Jonathan Tompson, Leonid Pishchulin, Micha Andriluka, Chris Bregler, Bernt Schiele, Christian Theobalt; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 3810-3818

Abstract


We present a novel method for accurate marker-less capture of articulated skeleton motion of several subjects in general scenes, indoors and outdoors, even from input filmed with as few as two cameras. Our approach unites a discriminative image-based joint detection method with a model-based generative motion tracking algorithm through a combined pose optimization energy. The discriminative part-based pose detection method, implemented using Convolutional Networks (ConvNet), estimates unary potentials for each joint of a kinematic skeleton model. These unary potentials are used to probabilistically extract pose constraints for tracking by using weighted sampling from a pose posterior guided by the model. In the final energy, these constraints are combined with an appearance-based model-to-image similarity term. Poses can be computed very efficiently using iterative local optimization, as ConvNet detection is fast, and our formulation yields a combined pose estimation energy with analytic derivatives. In combination, this enables to track full articulated joint angles at state-of-the-art accuracy and temporal stability with a very low number of cameras.

Related Material


[pdf]
[bibtex]
@InProceedings{Elhayek_2015_CVPR,
author = {Elhayek, Ahmed and de Aguiar, Edilson and Jain, Arjun and Tompson, Jonathan and Pishchulin, Leonid and Andriluka, Micha and Bregler, Chris and Schiele, Bernt and Theobalt, Christian},
title = {Efficient ConvNet-Based Marker-Less Motion Capture in General Scenes With a Low Number of Cameras},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2015}
}