We present a robust deep incremental learning framework for regression tasks on financial temporal tabular datasets which is built upon the incremental use of commonly available tabular and time series prediction models to adapt to distributional shifts typical of financial datasets. The framework uses a simple basic building block (decision trees) to build self-similar models of any required complexity to deliver robust performance under adverse situations such as regime changes, fat-tailed distributions, and low signal-to-noise ratios. As a detailed study, we demonstrate our scheme using XGBoost models trained on the Numerai dataset and show that a two layer deep ensemble of XGBoost models over different model snapshots delivers high quality predictions under different market regimes. We also show that the performance of XGBoost models with different number of boosting rounds in three scenarios (small, standard and large) is monotonically increasing with respect to model size and converges towards the generalisation upper bound. We also evaluate the robustness of the model under variability of different hyperparameters, such as model complexity and data sampling settings. Our model has low hardware requirements as no specialised neural architectures are used and each base model can be independently trained in parallel.

翻译：我们提出了一种稳健的深度增量学习框架，用于金融时序表格数据集上的回归任务。该框架基于增量式利用通用的表格模型和时间序列预测模型，以适应金融数据集典型的分布漂移。该框架使用简单的基本构建块（决策树）构建任意复杂度的自相似模型，从而在制度转换、厚尾分布和低信噪比等不利条件下实现稳健性能。作为详细研究，我们使用在Numerai数据集上训练的XGBoost模型演示了该方案，并表明：在不同模型快照上构建的两层深度XGBoost集成模型，能够在不同市场体制下提供高质量的预测。我们还证明，在三种场景（小规模、标准和大规模）下，具有不同提升轮数的XGBoost模型性能随模型规模单调递增，并收敛于泛化上界。此外，我们评估了模型在不同超参数（如模型复杂度和数据采样设置）变异下的稳健性。由于未使用专门神经架构且每个基模型可独立并行训练，我们的模型硬件需求较低。