Taming User-Interface Heterogeneity with Uniform Overlays for Blind Users

UMAP 2022

Utku Uckun1   Rohan Tumkur Suresh1   Md Javedul Ferdous2   Xiaojun Bi1   I.V. Ramakrishnan1   Vikas Ashok2

Stony Brook university1     Old Dominion University2

[Paper] [Bibtex]

feature

Abstract

For many blind users, interaction with computer applications using screen reader assistive technology is a frustrating and time-consuming affair, mostly due to the complexity and heterogeneity of applications’ user interfaces. An interview study revealed that many applications do not adequately convey their interface structure and controls to blind screen reader users, thereby placing additional burden on these users to acquire this knowledge on their own. This is often an arduous and tedious learning process given the one-dimensional navigation paradigm of screen readers. Moreover, blind users have to repeat this learning process multiple times, i.e., once for each application, since applications differ in their interface designs and implementations. In this paper, we propose a novel push-based approach to make non-visual computer interaction easy, efficient, and uniform across different applications. The key idea is to make screen reader interaction ‘structure-agnostic’, by automatically identifying and extracting all application controls and then instantly ‘pushing’ these controls on demand to the blind user via a custom overlay dashboard interface. Such a custom overlay facilitates uniform and efficient screen reader navigation across all applications. A user study showed significant improvement in user satisfaction and interaction efficiency with our approach compared to a state-of-the-art screen reader.

Publication

Uckun, Utku, et al. "Taming User-Interface Heterogeneity with Uniform Overlays for Blind Users." Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization. 2022.[Bibtex]

Acknowledgement

This work was supported by NSF Awards: 1805076, 1815514, 1936027, 2113485, 2125147 and NIH Awards: R01EY030085, R01HD097188.