No, we have not invented a new method to spy on you. Rather, we suggest a novel paradigm that helps usability experts understand in which situations users interacting with a rich Website progress smoothly and in which they don’t. The key to understanding users’ problems is summarising comparable situations that various users encounter at different points of their website journey.
Because of the complexity of modern Web GUIs, this requires the analyses of videos recording user interactions on websites. Our novel approach to discovering visual stimuli employs machine learning to judge the similarity between recorded situations. Our paper evaluates several case studies to demonstrate the effectiveness of our proposed method. It also surveys usability experts who confirm the usefulness of our suggested approach.
The paper results from a long-running collaboration between (i) Steffen Staab from Analytic Computing, (ii) Raphael Menges from Semanux GmbH, a spin-off from Analytic Computing, (iii) Christoph Schäfer and Tina Walber from EyeVido GmbH, a spin-off from Staab’s former research group at University of Koblenz and now a part of Tobii AB, the leading manufacturer of remote eye trackers, and (iv) Chandan Kumar, a team lead at Fraunhofer IAO, and previously a postdoc at Analytic Computing.
Cite the paper and find its download via:
Raphael Menges, Steffen Staab, Christoph Schaefer, Tina Walber, and Chandan Kumar. 2025. What Did My Users Experience? Discovering Visual Stimuli on Graphical User Interfaces of the Web. ACM Transactions on the Web (February 2025). https://doi.org/10.1145/3715881