Combining first-person and third-person gaze for attention recognition

Abstract

This paper presents a method to recognize attentional behaviors from a head-mounted binocular eye tracker in triadic interactions. By taking advantage of the first-person view, we simultaneously estimate the first-person and third-person gaze. The first-person gaze is computed using an appearance-based method relying on local features. In parallel, head pose tracking allows determining the coarse gaze of people in the scene camera. Finally, knowing the first- and third-person gaze direction, scores are computed which permit to assign attention patterns to each frame. Our contributions are the followings: (i) head pose estimation based on localized regression, (ii) attention analysis, in particular mutual and shared gaze, including the first-person gaze, (iii) experiments conducted using a head-mounted appearance-based gaze tracker. Experiments on recorded data show encouraging results.