In this paper, we explore the use of different feature engineering and dimensionality reduction methods in multi-variate time-series modelling. Using a feature-target cross correlation time series dataset created from Numerai tournament, we demonstrate under over-parameterised regime, both the performance and predictions from different feature engineering methods converge to the same equilibrium, which can be characterised by the reproducing kernel Hilbert space. We suggest a new Ensemble method, which combines different random non-linear transforms followed by ridge regression for modelling high dimensional time-series. Compared to some commonly used deep learning models for sequence modelling, such as LSTM and transformers, our method is more robust (lower model variance over different random seeds and less sensitive to the choice of architecture) and more efficient. An additional advantage of our method is model simplicity as there is no need to use sophisticated deep learning frameworks such as PyTorch. The learned feature rankings are then applied to the temporal tabular prediction problem in the Numerai tournament, and the predictive power of feature rankings obtained from our method is better than the baseline prediction model based on moving averages

翻译：本文探讨了多变量时间序列建模中不同特征工程和降维方法的应用。利用Numerai竞赛构建的特征-目标互相关时间序列数据集，我们发现在过参数化情况下，不同特征工程方法的性能和预测结果均收敛于同一均衡状态，该状态可通过再生核希尔伯特空间进行刻画。我们提出了一种新的集成方法，该方法结合了不同的随机非线性变换与岭回归，用于高维时间序列建模。与LSTM和Transformer等常用序列建模深度学习模型相比，本方法更具鲁棒性（在不同随机种子下模型方差更小，对架构选择的敏感性更低）且效率更高。本方法的另一优势在于模型简洁性，无需使用PyTorch等复杂深度学习框架。最后，我们将在Numerai竞赛中将学习到的特征排序应用于时序表格预测问题，实验表明本方法获得的特征排序预测能力优于基于移动平均的基线预测模型。