Hand shadow puppetry, also known as shadowgraphy or ombromanie, is a form of theatrical art and storytelling where hand shadows are projected onto flat surfaces to create illusions of living creatures. The skilled performers create these silhouettes by hand positioning, finger movements, and dexterous gestures to resemble shadows of animals and objects. Due to the lack of practitioners and a seismic shift in people's entertainment standards, this art form is on the verge of extinction. To facilitate its preservation and proliferate it to a wider audience, we introduce ${\rm H{\small A}SP{\small E}R}$, a novel dataset consisting of 15,000 images of hand shadow puppets across 15 classes extracted from both professional and amateur hand shadow puppeteer clips. We provide a detailed statistical analysis of the dataset and employ a range of pretrained image classification models to establish baselines. Our findings show a substantial performance superiority of skip-connected convolutional models over attention-based transformer architectures. We also find that lightweight models, such as MobileNetV2, suited for mobile applications and embedded devices, perform comparatively well. We surmise that such low-latency architectures can be useful in developing ombromanie teaching tools, and we create a prototype application to explore this surmission. Keeping the best-performing model ResNet34 under the limelight, we conduct comprehensive feature-spatial, explainability, and error analyses to gain insights into its decision-making process. To the best of our knowledge, this is the first documented dataset and research endeavor to preserve this dying art for future generations, with computer vision approaches. Our code and data will be publicly available.
翻译:手影戏,亦称光影艺术或手影幻术,是一种通过将手部阴影投射至平面以创造生物幻象的戏剧艺术与叙事形式。技艺精湛的表演者通过手部定位、手指运动及灵巧手势塑造出类似动物与物体轮廓的剪影。由于从业者稀缺及大众娱乐标准的根本性转变,该艺术形式正面临消亡危机。为促进其保护并推广至更广泛受众,我们提出${\rm H{\small A}SP{\small E}R}$——一个包含15个类别、共计15,000张手影图像的新型数据集,图像素材提取自专业及业余手影表演者的视频片段。我们对数据集进行了详细的统计分析,并采用一系列预训练图像分类模型建立性能基准。研究发现,跳跃连接卷积模型在性能上显著优于基于注意力机制的Transformer架构。同时发现适用于移动应用与嵌入式设备的轻量级模型(如MobileNetV2)表现出相对优越的性能。我们推断此类低延迟架构可用于开发手影教学工具,并创建了原型应用程序以验证该设想。聚焦性能最优的ResNet34模型,我们开展了全面的特征空间分析、可解释性分析与错误分析,以深入理解其决策机制。据我们所知,这是首个通过计算机视觉方法保护这一濒危艺术、并面向后世留存的数据集与研究实践。我们的代码与数据将公开提供。