We describe the development of a real-time smartphone app that allows the user to digitize paper receipts in a novel way by "waving" their phone over the receipts and letting the app automatically detect and rectify the receipts for subsequent text recognition. We show that traditional computer vision algorithms for edge and corner detection do not robustly detect the non-linear and discontinuous edges and corners of a typical paper receipt in real-world settings. This is particularly the case when the colors of the receipt and background are similar, or where other interfering rectangular objects are present. Inaccurate detection of a receipt's corner positions then results in distorted images when using an affine projective transformation to rectify the perspective. We propose an innovative solution to receipt corner detection by treating each of the four corners as a unique "object", and training a Single Shot Detection MobileNet object detection model. We use a small amount of real data and a large amount of automatically generated synthetic data that is designed to be similar to real-world imaging scenarios. We show that our proposed method robustly detects the four corners of a receipt, giving a receipt detection accuracy of 85.3% on real-world data, compared to only 36.9% with a traditional edge detection-based approach. Our method works even when the color of the receipt is virtually indistinguishable from the background. Moreover, our method is trained to detect only the corners of the central target receipt and implicitly learns to ignore other receipts, and other rectangular objects. Including synthetic data allows us to train an even better model. These factors are a major advantage over traditional edge detection-based approaches, allowing us to deliver a much better experience to the user.

翻译：我们描述了一款实时智能手机应用的开发过程，该应用通过新颖的方式让用户"挥动"手机扫描纸质票据，即可自动检测并纠偏票据，便于后续文本识别。研究表明，传统基于边缘与角点检测的计算机视觉算法在实际场景中无法稳健地检测典型纸质票据的非线性、非连续边缘及角点。当票据与背景颜色相近或存在其他干扰矩形物体时，该问题尤为突出。若票据角点位置检测不准确，使用仿射投影变换进行透视纠偏时便会产生变形图像。我们提出了一种创新的票据角点检测解决方案：将四个角点分别视为独立"目标"，并训练基于单次检测的MobileNet目标检测模型。我们结合少量真实数据与大量自动生成的、模拟真实成像场景的合成数据进行训练。实验表明，本方法能稳健检测票据四个角点，在真实数据上的票据检测准确率达85.3%，而传统基于边缘检测的方法仅为36.9%。即便票据颜色与背景几乎无法区分时，本方法依然有效。此外，该方法仅训练检测中心目标票据的角点，可隐式学习忽略其他票据及矩形物体。合成数据的引入进一步优化了模型性能。这些优势使本方法显著优于传统边缘检测方案，为用户提供了更优质的使用体验。