Many areas in science and engineering now have access to technologies that enable the rapid collection of overwhelming data volumes. While these datasets are vital for understanding phenomena from physical to biological and social systems, the sheer magnitude of the data makes even simple storage, transmission, and basic processing highly challenging. To enable efficient and accurate execution of these data processing tasks, we require new dimensionality reduction tools that 1) do not need expensive, time-consuming training, and 2) preserve the underlying geometry of the data that has the information required to understand the measured system. Specifically, the geometry to be preserved is that induced by the fact that in many applications, streaming high-dimensional data evolves on a low-dimensional attractor manifold. Importantly, we may not know the exact structure of this manifold a priori. To solve these challenges, we present randomized filtering (RF), which leverages a specific instantiation of randomized dimensionality reduction to provably preserve non-linear manifold structure in the embedded space while remaining data-independent and computationally efficient. In this work we build on the rich theoretical promise of randomized dimensionality reduction to develop RF as a real, practical approach. We introduce novel methods, analysis, and experimental verification to illuminate the practicality of RF in diverse scientific applications, including several simulated and real-data examples that showcase the tangible benefits of RF.

翻译：当前，科学与工程领域的许多学科已具备快速采集海量数据的技术能力。虽然这些数据集对于理解从物理到生物乃至社会系统的各类现象至关重要，但其庞大规模使得即使是简单的存储、传输与基础处理也极具挑战性。为实现高效且准确的数据处理任务，我们需要新型降维工具，其应满足：1）无需昂贵耗时的训练过程；2）保持数据底层几何结构，该结构蕴含着理解被测系统所需的信息。具体而言，需要保持的几何特性源于这样一个事实：在许多应用场景中，高维数据流演化于低维吸引子流形之上。值得注意的是，我们可能无法预先获知该流形的确切结构。为应对这些挑战，本文提出随机滤波方法，该方法通过特定形式的随机降维技术，在理论上保证嵌入空间中的非线性流形结构得以保持，同时保持数据无关性与计算高效性。本研究基于随机降维的丰富理论前景，将随机滤波发展为切实可行的实用方法。我们通过引入创新方法、理论分析与实验验证，阐明随机滤波在多样化科学应用中的实用性，包括多个模拟与真实数据案例，充分展示了该方法带来的实际效益。