We introduce a novel bottom-up approach for the extraction of chart data. Our model utilizes images of charts as inputs and learns to detect keypoints (KP), which are used to reconstruct the components within the plot area. Our novelty lies in detecting a fusion of continuous and discrete KP as predicted heatmaps. A combination of sparse and dense per-pixel objectives coupled with a uni-modal self-attention-based feature-fusion layer is applied to learn KP embeddings. Further leveraging deep metric learning for unsupervised clustering, allows us to segment the chart plot area into various objects. By further matching the chart components to the legend, we are able to obtain the data series names. A post-processing threshold is applied to the KP embeddings to refine the object reconstructions and improve accuracy. Our extensive experiments include an evaluation of different modules for KP estimation and the combination of deep layer aggregation and corner pooling approaches. The results of our experiments provide extensive evaluation for the task of real-world chart data extraction.

翻译：我们提出了一种新颖的自底向上方法用于图表数据提取。该模型以图表图像为输入，通过学习检测关键点来重构绘图区域内的组件。创新之处在于通过预测热力图检测连续与离散关键点的融合。结合稀疏和密集的逐像素目标函数，并应用基于单模态自注意力的特征融合层，我们学习了关键点嵌入。进一步利用深度度量学习进行无监督聚类，从而能够将图表绘图区域分割为不同对象。通过将图表组件与图例匹配，我们能够获取数据系列名称。对关键点嵌入应用后处理阈值以优化对象重构并提升精度。广泛的实验包括对不同关键点估计模块以及深度层聚合与角点池化组合方法的评估。实验结果为真实世界图表数据提取任务提供了全面的性能评测。