Title: Toward AI-Assisted UX Analysis Methods: Leveraging Subtle Verbalization and Speech Patterns to Help Evaluators Identify UX Problems in Think-Aloud Usability Testing 

Speaker: Mingming Fan (Rochester Institute of Technology) 

Time: 10:00 am. December 20th, 2019 

Venue: 5号楼四层中会议室 

　　Abstract: Pinpointing users’ pain points is key to improve user experience (UX). More than one trillion dollars, according to a recent report, will be lost due to bad UX in e-commerce alone by 2020. One common way to understand users’ pain points is to conduct usability testing using think-aloud protocols. Despite the value of conducting think-aloud usability testing, analyzing recorded test sessions is labor-intensive and time-consuming. Consequently, to effectively leverage the potential of a large amount of usability test sessions to improve UX, it is imperative to investigate methods to facilitate the analysis of usability test sessions. 

　　One important step toward this goal is to understand subtle patterns in users’ behavioral data that are indicative of UX problems. In this talk, I will present a series of user studies, each addressing the limitations of the previous one, to uncover subtle patterns in users’ verbalization and speech that are indicative of the UX problems that they encounter in think-aloud usability test sessions. Subsequently, I will introduce the computational methods that leverage the subtle patterns and machine learning techniques to automatically detect UX problems. Through an iterative design process, I further create an interactive intelligent analytical tool—VisTA—to visualize the ML’s predictions and its input features to assist UX practitioners with the analysis of recorded think-aloud sessions. I will present the design and evaluation of the tool and offer insights into how UX practitioners leverage and perceive ML-inferred problems and its input features in their analysis. Finally, I will highlight future directions to further advance AI-assisted UX analysis methods.  

　　Bio: Dr. Mingming Fan is an Assistant Professor in the Golisano College of Computing and Information Sciences at the Rochester Institute of Technology (RIT), USA. He received his Ph.D. in Computer Science from the University of Toronto in 2019 and an MSc in Computer Science with honor from Tsinghua University. He is a member of the Center for Accessibility and Inclusion Research (CAIR) and a member of the Center for Human-aware AI (CHAI) at RIT. He researches at the intersection of HCI and AI/ML, focusing on understanding subtle patterns in human behavioral signals that are indicative of user experience (UX) and leveraging such patterns to design AI-empowered interactive intelligent analytical systems. 

　　His research has published at top-tier conferences and journals in the field of HCI, UbiComp, and Accessibility, including ACM Transactions on Computer-Human Interaction (TOCHI), ACM CHI, ACM UbiComp, IEEE Transactions on Visualization and Computer Graphics (TVCG), ACM Transactions on Accessible Computing (TACCESS), and ACM ASSETS. His works have won a Best Paper Award from ACM CHI 2019 and a Best Paper Honorable Mention Award from ACM UbiComp 2015. More information about him and his research can be found on his website: http://mingmingfan.com/