Hierarchical learning algorithms that gradually approximate a solution to a data-driven optimization problem are essential to decision-making systems, especially under limitations on time and computational resources. In this study, we introduce a general-purpose hierarchical learning architecture that is based on the progressive partitioning of a possibly multi-resolution data space. The optimal partition is gradually approximated by solving a sequence of optimization sub-problems that yield a sequence of partitions with increasing number of subsets. We show that the solution of each optimization problem can be estimated online using gradient-free stochastic approximation updates. As a consequence, a function approximation problem can be defined within each subset of the partition and solved using the theory of two-timescale stochastic approximation algorithms. This simulates an annealing process and defines a robust and interpretable heuristic method to gradually increase the complexity of the learning architecture in a task-agnostic manner, giving emphasis to regions of the data space that are considered more important according to a predefined criterion. Finally, by imposing a tree structure in the progression of the partitions, we provide a means to incorporate potential multi-resolution structure of the data space into this approach, significantly reducing its complexity, while introducing hierarchical variable-rate feature extraction properties similar to certain classes of deep learning architectures. Asymptotic convergence analysis and experimental results are provided for supervised and unsupervised learning problems.

翻译：层次化学习算法通过逐步逼近数据驱动优化问题的解，对决策系统至关重要，尤其在时间和计算资源受限的情况下。本研究提出一种通用层次化学习架构，其核心思想是对可能具有多分辨率结构的数据空间进行渐进式划分。通过求解一系列优化子问题，逐步逼近最优划分，并生成子集数量递增的划分序列。我们证明了每个优化问题的解可通过无梯度随机逼近更新实现在线估计。在此基础上，可在划分的每个子集内定义函数逼近问题，并利用双时间尺度随机逼近算法理论求解。该过程模拟退火机制，构建了一种鲁棒且可解释的启发式方法，以与任务无关的方式逐步增加学习架构的复杂度，并依据预定准则重点关注数据空间中更重要的区域。最后，通过在划分的递进过程中引入树结构，我们提供了一种将数据空间潜在的多分辨率结构融入此方法的途径，显著降低了计算复杂度，同时引入了类似某些深度学习架构的层次化变速率特征提取特性。针对监督学习和无监督学习问题，本文提供了渐近收敛性分析及实验结果。