Quantifying biometric characteristics within hand gestures involve derivation of fitness scores from a gesture and identity aware feature space. However, evaluating the quality of these scores remains an open question. Existing biometric capacity estimation literature relies upon error rates. But these rates do not indicate goodness of scores. Thus, in this manuscript we present an exhaustive set of evaluation measures. We firstly identify ranking order and relevance of output scores as the primary basis for evaluation. In particular, we consider both rank deviation as well as rewards for: (i) higher scores of high ranked gestures and (ii) lower scores of low ranked gestures. We also compensate for correspondence between trends of output and ground truth scores. Finally, we account for disentanglement between identity features of gestures as a discounting factor. Integrating these elements with adequate weighting, we formulate advanced acceptance score as a holistic evaluation measure. To assess effectivity of the proposed we perform in-depth experimentation over three datasets with five state-of-the-art (SOTA) models. Results show that the optimal score selected with our measure is more appropriate than existing other measures. Also, our proposed measure depicts correlation with existing measures. This further validates its reliability. We have made our \href{https://github.com/AmanVerma2307/MeasureSuite}{code} public.
翻译:量化手势中的生物特征特性涉及从手势及身份感知特征空间中推导适应度评分。然而,评估这些评分的质量仍是一个开放性问题。现有的生物特征容量估计文献依赖于错误率,但这些错误率并不能反映评分的优劣。因此,本文提出了一套详尽的评估度量体系。我们首先将输出评分的排序顺序与相关性确定为评估的主要基础。具体而言,我们同时考虑了排序偏差以及对以下两方面的奖励:(i) 高排名手势获得较高评分,(ii) 低排名手势获得较低评分。我们还对输出评分与真实评分趋势之间的对应关系进行了补偿。最后,我们将手势身份特征之间的解耦作为折扣因子纳入考量。通过以适当权重整合这些要素,我们将高级接受度评分构建为一个整体性评估度量。为验证所提方法的有效性,我们在三个数据集上对五种最先进(SOTA)模型进行了深入实验。结果表明,采用我们的度量方法选出的最优评分比现有其他度量方法更为合理。同时,我们提出的度量与现有度量显示出相关性,这进一步验证了其可靠性。我们已将\href{https://github.com/AmanVerma2307/MeasureSuite}{代码}公开。