Web Science and Digital Libraries Research Group

  Figure 1: Example of AccessBolt with File Explorer application. (Figure 1 from Utku et al.) Introduction User interface (UI) automation is the process of programmatically controlling and testing the UI of applications. In our last blog on UI automation, we talk about the ideas of using it and walkthrough of these interesting tools using the Python programming language. The main purposes of UI automation are to simulate user interactions for testing, increase test coverage, and reduce repetitive manual testing. In this blog, we are going to talk about a recently published paper named “ Taming User-Interface Heterogeneity with Uniform Overlays for Blind Users, ” published at the UMAP 2022 conference, where Utku et al. explored how to improve the accessibility of desktop application interfaces for blind individuals who rely on screen readers . The paper introduces a novel system called AccessBolt that aims to make interacting with desktop software easier and more efficient using...