Given a huge, online stream of time-evolving events with multiple attributes, such as online shopping logs: (item, price, brand, time), and local mobility activities: (pick-up and drop-off locations, time), how can we summarize large, dynamic high-order tensor streams? How can we see any hidden patterns, rules, and anomalies? Our answer is to focus on two types of patterns, i.e., ''regimes'' and ''components'', for which we present CubeScope, an efficient and effective method over high-order tensor streams. Specifically, it identifies any sudden discontinuity and recognizes distinct dynamical patterns, ''regimes'' (e.g., weekday/weekend/holiday patterns). In each regime, it also performs multi-way summarization for all attributes (e.g., item, price, brand, and time) and discovers hidden ''components'' representing latent groups (e.g., item/brand groups) and their relationship. Thanks to its concise but effective summarization, CubeScope can also detect the sudden appearance of anomalies and identify the types of anomalies that occur in practice. Our proposed method has the following properties: (a) Effective: it captures dynamical multi-aspect patterns, i.e., regimes and components, and statistically summarizes all the events; (b) General: it is practical for successful application to data compression, pattern discovery, and anomaly detection on various types of tensor streams; (c) Scalable: our algorithm does not depend on the length of the data stream and its dimensionality. Extensive experiments on real datasets demonstrate that CubeScope finds meaningful patterns and anomalies correctly, and consistently outperforms the state-of-the-art methods as regards accuracy and execution speed.
翻译:针对包含多重属性的大规模在线事件流,例如在线购物日志:(商品、价格、品牌、时间) 以及本地移动活动:(上下车地点、时间),我们如何总结大规模动态高阶张量流?如何发现其中隐藏的模式、规则与异常?我们的答案是聚焦于两类模式,即“状态”与“成分”,并为此提出一种高效且有效的高阶张量流处理方法CubeScope。具体而言,它能识别任何突发的不连续性,区分不同的动态模式即“状态”(例如工作日/周末/节假日模式)。在每个状态内,它还能对所有属性(如商品、价格、品牌和时间)进行多面向总结,并发现代表潜在组群(如商品/品牌组)及其关系的隐藏“成分”。凭借其简洁而有效的总结能力,CubeScope还可检测异常的突然出现,并识别实践中发生的异常类型。我们提出的方法具有以下特性:(a) 有效性:它捕捉动态多面向模式,即状态与成分,并对所有事件进行统计总结;(b) 通用性:它可实际应用于多种张量流类型的数据压缩、模式发现与异常检测;(c) 可扩展性:我们的算法不依赖于数据流的长度及其维度。在真实数据集上的大量实验表明,CubeScope能正确发现有意义的模式与异常,并在准确率和执行速度上持续优于现有最先进方法。