Screen recordings of mobile applications are easy to capture and include a wealth of information, making them a popular mechanism for users to inform developers of the problems encountered in the bug reports. However, watching the bug recordings and efficiently understanding the semantics of user actions can be time-consuming and tedious for developers. Inspired by the conception of the video subtitle in movie industry, we present a lightweight approach CAPdroid to caption bug recordings automatically. CAPdroid is a purely image-based and non-intrusive approach by using image processing and convolutional deep learning models to segment bug recordings, infer user action attributes, and generate subtitle descriptions. The automated experiments demonstrate the good performance of CAPdroid in inferring user actions from the recordings, and a user study confirms the usefulness of our generated step descriptions in assisting developers with bug replay.

翻译：移动应用程序的屏幕录制易于捕捉且包含丰富信息，这使其成为用户向开发者告知错误报告中遇到问题的常用机制。然而，观看错误录制视频并高效理解用户操作的语义对开发者而言既耗时又繁琐。受电影行业视频字幕概念的启发，我们提出了一种轻量级方法CAPdroid，可自动为错误录制视频添加字幕。CAPdroid是一种纯图像驱动的非侵入式方法，通过图像处理技术和卷积深度学习模型来分割错误录制视频、推断用户操作属性并生成字幕描述。自动化实验证明了CAPdroid在从录制视频中推断用户操作方面的良好性能，而用户研究则确认了我们生成的步骤描述在协助开发者复现错误时的实用性。