Recent works have shown that deep learning (DL) models can effectively learn city-wide crowd-flow patterns, which can be used for more effective urban planning and smart city management. However, DL models have been known to perform poorly on inconspicuous adversarial perturbations. Although many works have studied these adversarial perturbations in general, the adversarial vulnerabilities of deep crowd-flow prediction models in particular have remained largely unexplored. In this paper, we perform a rigorous analysis of the adversarial vulnerabilities of DL-based crowd-flow prediction models under multiple threat settings, making three-fold contributions. (1) We propose CaV-detect by formally identifying two novel properties - Consistency and Validity - of the crowd-flow prediction inputs that enable the detection of standard adversarial inputs with 0% false acceptance rate (FAR). (2) We leverage universal adversarial perturbations and an adaptive adversarial loss to present adaptive adversarial attacks to evade CaV-detect defense. (3) We propose CVPR, a Consistent, Valid and Physically-Realizable adversarial attack, that explicitly inducts the consistency and validity priors in the perturbation generation mechanism. We find out that although the crowd-flow models are vulnerable to adversarial perturbations, it is extremely challenging to simulate these perturbations in physical settings, notably when CaV-detect is in place. We also show that CVPR attack considerably outperforms the adaptively modified standard attacks in FAR and adversarial loss metrics. We conclude with useful insights emerging from our work and highlight promising future research directions.

翻译：近期研究表明，深度学习模型能够有效学习城市范围的人群流量模式，可用于更高效的城市规划与智慧城市管理。然而，已知深度学习模型对不易察觉的对抗扰动表现欠佳。尽管已有大量研究探讨了常见的对抗扰动，但深度人群流量预测模型特有的对抗脆弱性仍鲜有探索。本文在多种威胁场景下对基于深度学习的人群流量预测模型的对抗脆弱性进行了严谨分析，并作出三方面贡献：（1）通过形式化定义人群流量预测输入的两个新特性——一致性与有效性——提出CaV-detect检测方法，实现对标准对抗输入的零错误接受率检测；（2）利用通用对抗扰动与自适应对抗损失，提出可规避CaV-detect防御的自适应对抗攻击；（3）提出CVPR攻击方法，这是一种兼具一致性、有效性与物理可实现性的对抗攻击，通过在扰动生成机制中显式引入一致性与有效性先验知识。研究发现，尽管人群流量模型易受对抗扰动影响，但在物理场景中模拟这些扰动极具挑战性，尤其当采用CaV-detect防御时。实验表明，CVPR攻击在错误接受率与对抗损失指标上显著优于经适应性改进的标准攻击。最后，本文总结研究启示并展望未来研究方向。