We consider the sequential decision-making problem where the mean outcome is a non-linear function of the chosen action. Compared with the linear model, two curious phenomena arise in non-linear models: first, in addition to the "learning phase" with a standard parametric rate for estimation or regret, there is an "burn-in period" with a fixed cost determined by the non-linear function; second, achieving the smallest burn-in cost requires new exploration algorithms. For a special family of non-linear functions named ridge functions in the literature, we derive upper and lower bounds on the optimal burn-in cost, and in addition, on the entire learning trajectory during the burn-in period via differential equations. In particular, a two-stage algorithm that first finds a good initial action and then treats the problem as locally linear is statistically optimal. In contrast, several classical algorithms, such as UCB and algorithms relying on regression oracles, are provably suboptimal.

翻译：我们考虑序贯决策问题，其中平均结果是所选动作的非线性函数。与线性模型相比，非线性模型中出现两个奇特现象：首先，除了具有标准参数速率的学习阶段（用于估计或遗憾）外，还存在一个由非线性函数确定的固定成本的“预热期”；其次，实现最小预热成本需要新的探索算法。针对文献中称为岭函数的特殊非线性函数族，我们推导了最优预热成本的上下界，并进一步通过微分方程刻画了预热期内的完整学习轨迹。特别地，一种先找到良好初始动作、再将问题视为局部线性处理的两阶段算法在统计意义上是最优的。相比之下，经典算法（如UCB及依赖回归黑箱的算法）被证明是次优的。