Analyzing user behavior from usability evaluation can be a challenging and time-consuming task, especially as the number of participants and the scale and complexity of the evaluation grows. We propose uxSense, a visual analytics system using machine learning methods to extract user behavior from audio and video recordings as parallel time-stamped data streams. Our implementation draws on pattern recognition, computer vision, natural language processing, and machine learning to extract user sentiment, actions, posture, spoken words, and other features from such recordings. These streams are visualized as parallel timelines in a web-based front-end, enabling the researcher to search, filter, and annotate data across time and space. We present the results of a user study involving professional UX researchers evaluating user data using uxSense. In fact, we used uxSense itself to evaluate their sessions.

翻译：从可用性评估中分析用户行为可能是一项具挑战性且耗时的任务，尤其是随着参与者数量增加以及评估的规模与复杂性提升。我们提出 uxSense，这是一个利用机器学习方法从音频和视频记录中提取用户行为作为并行时间戳数据流的可视分析系统。我们的实现融合了模式识别、计算机视觉、自然语言处理和机器学习，从这些记录中提取用户情感、动作、姿态、口语内容及其他特征。这些数据流在基于Web的前端中以并行时间线的形式可视化，使研究人员能够跨时间与空间搜索、筛选和标注数据。我们展示了一项用户研究的结果，该研究涉及专业用户体验研究人员使用uxSense评估用户数据。事实上，我们使用uxSense本身来评估他们的分析过程。