Discovering Objects of Joint Attention via First-Person Sensing

Hiroshi Kera, Ryo Yonetani, Keita Higuchi, Yoichi Sato; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2016, pp. 7-15

Abstract


The goal of this work is to discover objects of joint attention, i.e., objects being viewed by multiple people using head-mounted cameras and eye trackers. Such objects of joint attention are expected to act as an important cue for understanding social interactions in everyday scenes. To this end, we develop a commonality-clustering method tailored to first-person videos combined with points-of-gaze sources. The proposed method uses multiscale spatiotemporal tubes around points of gaze as a candidate of objects, making it possible to deal with various sizes of objects observed in the first-person videos. We also introduce a new dataset of multiple pairs of first-person videos and points-of-gaze data. Our experimental results show that our approach can outperform several state-of-the-art commonality-clustering methods.

Related Material


[pdf]
[bibtex]
@InProceedings{Kera_2016_CVPR_Workshops,
author = {Kera, Hiroshi and Yonetani, Ryo and Higuchi, Keita and Sato, Yoichi},
title = {Discovering Objects of Joint Attention via First-Person Sensing},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2016}
}