Motivated by programmatic advertising optimization, we consider the task of sequentially allocating budget across a set of resources. At every time step, a feasible allocation is chosen and only a corresponding random return is observed. The goal is to maximize the cumulative expected sum of returns. This is a realistic model for budget allocation across subdivisions of marketing campaigns, with the objective of maximizing the number of conversions. We study direct search (also known as pattern search) methods for linearly constrained and derivative-free optimization in the presence of noise, which apply in particular to sequential budget allocation. These algorithms, which do not rely on hierarchical partitioning of the resource space, are easy to implement; they respect the operational constraints of resource allocation by avoiding evaluation outside of the feasible domain; and they are also compatible with warm start by being (approximate) descent algorithms. However, they have not yet been analyzed from the perspective of cumulative regret. We show that direct search methods achieves finite regret in the deterministic and unconstrained case. In the presence of evaluation noise and linear constraints, we propose a simple extension of direct search that achieves a regret upper-bound of the order of $T^{2/3}$. We also propose an accelerated version of the algorithm, relying on repeated sequential testing, that significantly improves the practical behavior of the approach.

翻译：受程序化广告优化的启发，我们考虑在资源集合中顺序分配预算的任务。在每个时间步，选择一个可行的分配方案，并仅观察到相应的随机回报。目标是最大化累积期望回报总和。这是营销活动各细分领域预算分配的现实模型，其目标是最大化转化次数。我们研究了存在噪声情况下线性约束和无导数优化的直接搜索（也称为模式搜索）方法，这些方法特别适用于顺序预算分配。这些算法不依赖于资源空间的层次划分，易于实现；它们通过避免在可行域外进行评估来满足资源分配的操作约束；并且作为（近似）下降算法，它们也兼容热启动。然而，尚未从累积遗憾的角度对其进行分析。我们证明，在确定性和无约束情况下，直接搜索方法可实现有限遗憾。在存在评估噪声和线性约束的情况下，我们提出直接搜索的简单扩展，其遗憾上界为$T^{2/3}$量级。我们还提出该算法的加速版本，依赖于重复顺序测试，显著改善了方法的实际表现。