Hand shadow puppetry, also known as shadowgraphy or ombromanie, is a form of theatrical art and storytelling where hand shadows are projected onto flat surfaces to create illusions of living creatures. The skilled performers create these silhouettes by hand positioning, finger movements, and dexterous gestures to resemble shadows of animals and objects. Due to the lack of practitioners and a seismic shift in people's entertainment standards, this art form is on the verge of extinction. To facilitate its preservation and proliferate it to a wider audience, we introduce ${\rm H{\small A}SP{\small E}R}$, a novel dataset consisting of 15,000 images of hand shadow puppets across 15 classes extracted from both professional and amateur hand shadow puppeteer clips. We provide a detailed statistical analysis of the dataset and employ a range of pretrained image classification models to establish baselines. Our findings show a substantial performance superiority of skip-connected convolutional models over attention-based transformer architectures. We also find that lightweight models, such as MobileNetV2, suited for mobile applications and embedded devices, perform comparatively well. We surmise that such low-latency architectures can be useful in developing ombromanie teaching tools, and we create a prototype application to explore this surmission. Keeping the best-performing model ResNet34 under the limelight, we conduct comprehensive feature-spatial, explainability, and error analyses to gain insights into its decision-making process. To the best of our knowledge, this is the first documented dataset and research endeavor to preserve this dying art for future generations, with computer vision approaches. Our code and data will be publicly available.
翻译:手影戏,亦称光影艺术或手影幻术,是一种通过将手部阴影投射至平面以创造生物幻象的剧场艺术与叙事形式。技艺精湛的表演者通过手部定位、手指运动及灵巧姿态塑造出类似动物与物体轮廓的剪影。由于从业者稀缺及大众娱乐标准的根本性转变,该艺术形式正面临消亡危机。为促进其保护工作并扩大受众群体,我们推出${\rm H{\small A}SP{\small E}R}$——一个包含15个类别、共15,000张手影图像的新型数据集,所有图像均从专业及业余手影表演者的影像素材中提取。我们对数据集进行了详细的统计分析,并采用一系列预训练图像分类模型建立性能基准。研究发现,跳跃连接卷积模型在性能上显著优于基于注意力机制的Transformer架构。同时发现适用于移动应用与嵌入式设备的轻量级模型(如MobileNetV2)表现相对优异。我们推断此类低延迟架构可用于开发手影教学工具,并创建了原型应用以验证此设想。聚焦性能最优的ResNet34模型,我们开展了全面的特征空间分析、可解释性分析与错误分析,以深入理解其决策机制。据我们所知,这是首个通过计算机视觉方法保护这一濒危艺术、并完整记录数据集与研究过程的学术工作。我们的代码与数据将公开提供。