We propose a novel framework to evaluate the robustness of transformer-based form field extraction methods via form attacks. We introduce 14 novel form transformations to evaluate the vulnerability of the state-of-the-art field extractors against form attacks from both OCR level and form level, including OCR location/order rearrangement, form background manipulation and form field-value augmentation. We conduct robustness evaluation using real invoices and receipts, and perform comprehensive research analysis. Experimental results suggest that the evaluated models are very susceptible to form perturbations such as the variation of field-values (~15% drop in F1 score), the disarrangement of input text order(~15% drop in F1 score) and the disruption of the neighboring words of field-values(~10% drop in F1 score). Guided by the analysis, we make recommendations to improve the design of field extractors and the process of data collection.

翻译：我们提出一个新的框架,以评价以变压器为基础的以形式为基础的实地抽取方法通过形式攻击的稳健性。我们引入了14种新形式变换,以评价最先进的实地抽取器在从OCR级别和形式级别(包括OCR位置/顺序重新安排、背景操作和外地价值增殖形式)受到形式攻击的脆弱性。我们利用真实的发票和收据进行稳健性评价,并进行全面的研究分析。实验结果表明,经评估的模型非常容易形成干扰,例如外地价值的变化(F1分下降15%)、输入文本顺序脱序(F1分下降15%)和外地价值相邻词的中断(F1分下降10% ) 。根据分析,我们提出了改进外地提取器设计和数据收集过程的建议。