Spatiotemporal Analysis of RGB-D-T Facial Images for Multimodal Pain Level Recognition

Ramin Irani, Kamal Nasrollahi, Marc O. Simon, Ciprian A. Corneanu, Sergio Escalera, Chris Bahnsen, Dennis H. Lundtoft, Thomas B. Moeslund, Tanja L. Pedersen, Maria-Louise Klitgaard, Laura Petrini; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2015, pp. 88-95

Abstract

Pain is a vital sign of human health and its automatic detection can be of crucial importance in many different contexts, including medical scenarios. While most available computer vision techniques are based on RGB, in this paper, we investigate the effect of combining RGB, depth, and thermal facial images for pain detection and pain intensity level recognition. For this purpose, we extract energies released by facial pixels using a spatiotemporal filter. Experiments on a group of 12 elderly people applying the multimodal approach show that the proposed method successfully detects pain and recognizes between three intensity levels in 82% of the analyzed frames improving more than 6% over RGB only analysis in similar conditions.

Related Material

[pdf]
[bibtex]
@InProceedings{Irani_2015_CVPR_Workshops,
author = {Irani, Ramin and Nasrollahi, Kamal and Simon, Marc O. and Corneanu, Ciprian A. and Escalera, Sergio and Bahnsen, Chris and Lundtoft, Dennis H. and Moeslund, Thomas B. and Pedersen, Tanja L. and Klitgaard, Maria-Louise and Petrini, Laura},
title = {Spatiotemporal Analysis of RGB-D-T Facial Images for Multimodal Pain Level Recognition},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2015}
}