Reconstructing user's attention on the web through mouse movements and perception-based content identification

Eye tracking is one of the most exploited techniques in literature for finding usability problems in web-based user interfaces (UIs). However, it is usually employed in a laboratory setting, considering that an eye-tracker is not commonly used in web browsing. In contrast, web application providers usually exploit remote techniques for large-scale user studies (e.g. A/B testing), tracking low-level interactions such as mouse clicks and movements. In this article, we discuss a method for predicting whether the user is looking at the content pointed by the cursor, exploiting the mouse movement data and a segmentation of the contents in a web page. We propose an automatic method for segmenting content groups inside a web page that, applying both image and code analysis techniques, identifies the user-perceived group of contents with a mean pixel-based error around the 20%. In addition, we show through a user study that such segmentation information enhances the precision and the accuracy in predicting the correlation between between the user's gaze and the mouse position at the content level, without relaying on user-specific features.