Combining conversational AI with refreshable tactile displays (RTDs) offers significant potential for creating accessible data visualization for people who are blind or have low vision (BLV). To support researchers and developers building accessible data visualizations with RTDs, we present a multimodal data interaction architecture along with an open-source reference implementation. Our system is the first to combine touch input with a conversational agent on an RTD, enabling deictic queries that fuse touch context with spoken language, such as "what is the trend between these points?" The architecture addresses key technical challenges, including touch sensing on RTDs, visual-to-tactile encoding, integrating touch context with conversational AI, and synchronizing multimodal output. Our contributions are twofold: (1) a technical architecture integrating RTD hardware, external touch sensing, and conversational AI to enable multimodal data interaction; and (2) an open-source reference implementation demonstrating its feasibility. This work provides a technical foundation to support future research in multimodal accessible data visualization.

翻译：将对话式AI与可刷新触觉显示器（RTD）相结合，为盲人或低视力（BLV）人士创建无障碍数据可视化提供了巨大潜力。为支持研究人员和开发者利用RTD构建无障碍数据可视化，我们提出了一种多模态数据交互架构及其开源参考实现。我们的系统首次在RTD上结合了触控输入与对话代理，支持融合触控上下文与自然语言的指示性查询，例如“这两点之间的趋势是什么？”。该架构解决了多项关键技术挑战，包括RTD上的触觉传感、视觉到触觉的编码、触控上下文与对话式AI的集成，以及多模态输出的同步。我们的贡献包括：（1）集成RTD硬件、外部触觉传感与对话式AI以实现多模态数据交互的技术架构；（2）证明其可行性的开源参考实现。这项工作为未来多模态无障碍数据可视化的研究提供了技术基础。