Hierarchical learning algorithms that gradually approximate a solution to a data-driven optimization problem are essential to decision-making systems, especially under limitations on time and computational resources. In this study, we introduce a general-purpose hierarchical learning architecture that is based on the progressive partitioning of a possibly multi-resolution data space. The optimal partition is gradually approximated by solving a sequence of optimization sub-problems that yield a sequence of partitions with increasing number of subsets. We show that the solution of each optimization problem can be estimated online using gradient-free stochastic approximation updates. As a consequence, a function approximation problem can be defined within each subset of the partition and solved using the theory of two-timescale stochastic approximation algorithms. This simulates an annealing process and defines a robust and interpretable heuristic method to gradually increase the complexity of the learning architecture in a task-agnostic manner, giving emphasis to regions of the data space that are considered more important according to a predefined criterion. Finally, by imposing a tree structure in the progression of the partitions, we provide a means to incorporate potential multi-resolution structure of the data space into this approach, significantly reducing its complexity, while introducing hierarchical variable-rate feature extraction properties similar to certain classes of deep learning architectures. Asymptotic convergence analysis and experimental results are provided for supervised and unsupervised learning problems.

翻译：分层学习算法通过逐步逼近数据驱动优化问题的解，对决策系统至关重要，尤其在时间与计算资源受限的情况下。本研究提出一种通用分层学习架构，其基础是对可能的多分辨率数据空间进行渐进划分。通过求解一系列优化子问题，逐步逼近最优划分，生成子集数量递增的分区序列。我们证明，每个优化问题的解可通过无梯度随机逼近更新在线估计。由此，可在每个分区子集内定义函数逼近问题，并利用双时间尺度随机逼近算法理论求解。此过程模拟退火机制，定义了一种鲁棒且可解释的启发式方法，以任务无关方式逐步提升学习架构的复杂度，同时依据预设准则重点关注数据空间中更重要的区域。最后，通过为分区演进步骤施加树结构，我们可在该方法中融入数据空间潜在的多分辨率结构，显著降低其复杂度，并引入类似特定深度学习架构的分层变速率特征提取特性。本文针对有监督与无监督学习问题，提供了渐近收敛性分析及实验结果。