A soft-max function has two main efficiency measures: (1) approximation - which corresponds to how well it approximates the maximum function, (2) smoothness - which shows how sensitive it is to changes of its input. Our goal is to identify the optimal approximation-smoothness tradeoffs for different measures of approximation and smoothness. This leads to novel soft-max functions, each of which is optimal for a different application. The most commonly used soft-max function, called exponential mechanism, has optimal tradeoff between approximation measured in terms of expected additive approximation and smoothness measured with respect to Rényi Divergence. We introduce a soft-max function, called "piecewise linear soft-max", with optimal tradeoff between approximation, measured in terms of worst-case additive approximation and smoothness, measured with respect to $\ell_q$-norm. The worst-case approximation guarantee of the piecewise linear mechanism enforces sparsity in the output of our soft-max function, a property that is known to be important in Machine Learning applications [Martins et al. '16, Laha et al. '18] and is not satisfied by the exponential mechanism. Moreover, the $\ell_q$-smoothness is suitable for applications in Mechanism Design and Game Theory where the piecewise linear mechanism outperforms the exponential mechanism. Finally, we investigate another soft-max function, called power mechanism, with optimal tradeoff between expected \textit{multiplicative} approximation and smoothness with respect to the Rényi Divergence, which provides improved theoretical and practical results in differentially private submodular optimization.

翻译：Soft-Max函数具有两个主要效率度量指标：(1) 逼近度——反映其对最大值函数的近似能力；(2) 光滑性——表征其对于输入变化的敏感程度。本文旨在针对不同的逼近度与光滑性度量方式，确定最优的逼近-光滑性权衡关系。由此我们推导出多种新型Soft-Max函数，每种函数在不同应用场景中均具有最优特性。最常用的Soft-Max函数（称为指数机制）在期望加性逼近度与基于Rényi散度的光滑性度量之间达到了最优权衡。我们提出一种称为"分段线性Soft-Max"的新型函数，在最坏情况加性逼近度与基于$\ell_q$范数的光滑性度量之间实现了最优权衡。该分段线性机制的最坏情况逼近保证能够强制Soft-Max函数输出具有稀疏性——这一特性在机器学习应用中至关重要[Martins等人'16, Laha等人'18]，而指数机制无法满足该特性。此外，$\ell_q$光滑性适用于机制设计与博弈论领域，在这些场景中分段线性机制的表现优于指数机制。最后，我们研究了另一种称为幂机制的Soft-Max函数，该函数在期望乘性逼近度与基于Rényi散度的光滑性之间达到最优权衡，为差分隐私下的子模优化问题提供了更优的理论与实用结果。