Skeleton-based action recognition receives the attention of many researchers as it is robust to viewpoint and illumination changes, and its processing is much more efficient than the processing of video frames. With the emergence of deep learning models, it has become very popular to represent the skeleton data in pseudo-image form and apply CNN for action recognition. Thereafter, studies concentrated on finding effective methods for forming pseudo-images. Recently, attention networks, more specifically transformers have provided promising results in various vision problems. In this study, the effectiveness of VIT for skeleton-based action recognition is examined and its robustness on the pseudo-image representation scheme is investigated. To this end, a three-level architecture, SkelVit is proposed, which forms a set of pseudo images, applies a classifier on each of the representations, and combines their results to find the final action class. The performance of SkelVit is examined thoroughly via a set of experiments. First, the sensitivity of the system to representation is investigated by comparing it with two of the state-of-the-art pseudo-image representation methods. Then, the classifiers of SkelVit are realized in two experimental setups by CNNs and VITs, and their performances are compared. In the final experimental setup, the contribution of combining classifiers is examined by applying the model with a different number of classifiers. Experimental studies reveal that the proposed system with its lightweight representation scheme achieves better results than the state-of-the-art methods. It is also observed that the vision transformer is less sensitive to the initial pseudo-image representation compared to CNN. Nevertheless, even with the vision transformer, the recognition performance can be further improved by the consensus of classifiers.

翻译：骨架动作识别因对视角和光照变化具有鲁棒性，且处理效率远高于视频帧分析，受到众多研究者的关注。随着深度学习模型的出现，将骨架数据以伪图像形式表示并应用CNN进行动作识别的方法变得非常流行。此后，研究重点集中于寻找有效的伪图像生成方法。近年来，注意力网络——特别是Transformer——在多种视觉问题中展现出优异性能。本研究探索了VIT在骨架动作识别中的有效性，并考察其对伪图像表示方案的鲁棒性。为此，我们提出三层体系结构SkelVit：首先生成一组伪图像，然后对每种表示应用分类器，最后融合各分类器结果以确定最终动作类别。通过系列实验全面评估了SkelVit的性能：首先，通过对比两种最先进的伪图像表示方法，检验系统对表示方式的敏感性；其次，在两组实验配置中分别采用CNN和VIT实现SkelVit分类器并比较性能；最后，通过使用不同数量的分类器验证多分类器融合的贡献。实验表明，所提系统凭借轻量级表示方案取得了优于现有方法的结果。同时发现，与CNN相比，视觉Transformer对初始伪图像表示的敏感性较低。然而，即使采用视觉Transformer，通过分类器共识仍可进一步提升识别性能。