Probabilistic Elastic Matching for Pose Variant Face Verification

Haoxiang Li, Gang Hua, Zhe Lin, Jonathan Brandt, Jianchao Yang; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pp. 3499-3506


Pose variation remains to be a major challenge for realworld face recognition. We approach this problem through a probabilistic elastic matching method. We take a part based representation by extracting local features (e.g., LBP or SIFT) from densely sampled multi-scale image patches. By augmenting each feature with its location, a Gaussian mixture model (GMM) is trained to capture the spatialappearance distribution of all face images in the training corpus. Each mixture component of the GMM is confined to be a spherical Gaussian to balance the influence of the appearance and the location terms. Each Gaussian component builds correspondence of a pair of features to be matched between two faces/face tracks. For face verification, we train an SVM on the vector concatenating the difference vectors of all the feature pairs to decide if a pair of faces/face tracks is matched or not. We further propose a joint Bayesian adaptation algorithm to adapt the universally trained GMM to better model the pose variations between the target pair of faces/face tracks, which consistently improves face verification accuracy. Our experiments show that our method outperforms the state-ofthe-art in the most restricted protocol on Labeled Face in the Wild (LFW) and the YouTube video face database by a significant margin.

Related Material

author = {Li, Haoxiang and Hua, Gang and Lin, Zhe and Brandt, Jonathan and Yang, Jianchao},
title = {Probabilistic Elastic Matching for Pose Variant Face Verification},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2013}