Michael Barz and Professor Daniel Sonntag, in collaboration with Professor Cristina Conati from The University of British Columbia, hosted the inaugural HumanEYEze Workshop [1]. This event aimed to unite researchers from the fields of eye-tracking, multimodal human-computer interaction, and artificial intelligence. It was held on Monday, November 4, 2024, and Read more…
At the 26th International Conference on Multimodal Interaction (ICMI 2024) in San José, Costa Rica, IML had several notable contributions. A full paper titled “Perceived Text Relevance Estimation Using Scanpaths and GNNs,” by Abdulrahman Mohamed [1], a workshop paper titled “Detecting when Users Disagree with Generated Captions,” by Omair Shahzad Read more…
In November, the EU Project MASTER held its second General Assembly and Technical Review meeting, and László Kopácsi and Michael Barz from IML were part of it. They met at Virtualware in Bilbao, Spain, for insightful discussions and engaging demos, and the consortium presented its progress to EU evaluators. The Read more…
Motivation The usage of interactive public displays has increased including the number of sensitive applications and, hence, the demand for user authentication methods. We implement a calibration-free authentication method for situated displays based on saccadic eye movements which tackles the drawbacks of gaze-based authentications: they tend to be slow and Read more…
Visual Search Target Inference in Natural Interaction Settings with Machine Learning Visual search is a perceptual task in which humans aim at identifying a search target object such as a traffic sign among other objects. Search target inference subsumes computational methods for predicting this target by tracking and analyzing overt Read more…
Digital pen signals were shown to be predictive for cognitive states, cognitive load and emotion in educational settings. We investigate whether low-level pen-based features can predict the difficulty of tasks in a cognitive test and the learner’s performance in these tasks, which is inherently related to cognitive load, without a Read more…
We implement a method for re-ranking top-10 results of a state-of-the-art question answering (QA) system. The goal of our re-ranking approach is to improve the answer selection given the user question and the top-10 candidates. We focus on improving deployed QA systems that do not allow re-training or when re-training Read more…
In this project, we consider scenarios which include one or more expert users that interact with a multimodal-multisensor interface. In particular, we focus on applications of eye tracking and speech-based input for semantic search applications, e.g., interactive knowledge exploration. We aim at developing interaction techniques for individual modalities, combinations thereof Read more…
Gaze estimation error can severely hamper usability and performance of mobile gaze-based interfaces given that the error varies constantly for different interaction positions. In this work, we explore error-aware gaze-based interfaces that estimate and adapt to gaze estimation error on-the-fly. We implement a sample error-aware user interface for gaze-based selection and Read more…
Visual Attention and the Artificial Episodic Memory Recent advances in mobile eye tracking technologies opened the way to design novel attention-aware intelligent user interaction. In particular, the gaze signal can be used to create or improve artificial episodic memories for offline processing, but also real-time processing. We investigate different methods Read more…