Pose and Expression-Coherent Face Recovery in the Wild

Xavier P. Burgos-Artizzu, Joaquin Zepeda, Francois Le Clerc, Patrick Perez; The IEEE International Conference on Computer Vision (ICCV) Workshops, 2015, pp. 37-45


We present a novel method to recover images of faces, particularly when large spatial regions of the face are unavailable due to data losses or occlusions. In contrast with previous work, we do not make assumptions on the data neither during training nor testing (such as assuming that the person was seen before or that all faces are perfectly aligned and have identical head pose, expression, etc.). Instead, we propose to tackle the problem in a purely unsupervised way, leveraging a large face dataset. During training, first we cluster faces based on their landmark's positions (obtained by an automatic face landmark estimator). Then, we model the face appearance for each group using sparse coding with learned dictionaries, with one dictionary per cluster. At test time, given a face to recover, we find its belonging cluster and occluded area and restore missing pixels by applying the group-specific sparse appearance representation learned during training. We show results on two "in the wild" datasets. Our method shows promising results on challenging faces and our sparse coding approach outperforms prior subspace learning techniques.

Related Material

author = {Burgos-Artizzu, Xavier P. and Zepeda, Joaquin and Le Clerc, Francois and Perez, Patrick},
title = {Pose and Expression-Coherent Face Recovery in the Wild},
booktitle = {The IEEE International Conference on Computer Vision (ICCV) Workshops},
month = {December},
year = {2015}