Accuracy at the top is a special class of binary classification problems where the performance is evaluated only on a small number of relevant (top) samples. Applications include information retrieval systems or processes with manual (expensive) postprocessing. This leads to minimizing the number of irrelevant samples above a threshold. We consider classifiers in the form of an arbitrary (deep) network and propose a new method DeepTopPush for minimizing the loss function at the top. Since the threshold depends on all samples, the problem is non-decomposable. We modify the stochastic gradient descent to handle the non-decomposability in an end-to-end training manner and propose a way to estimate the threshold only from values on the current minibatch and one delayed value. We demonstrate the excellent performance of DeepTopPush on visual recognition datasets and two real-world applications. The first one selects a small number of molecules for further drug testing. The second one uses real malware data, where we detected 46\% malware at an extremely low false alarm rate of $10^{-5}$.

翻译：顶部精度是一类特殊的二元分类问题，其中性能仅基于少量相关（顶部）样本进行评估。其应用包括信息检索系统或需要人工（昂贵）后处理的流程。这导致需要将超过某个阈值的不相关样本数量最小化。我们考虑基于任意（深层）网络的分类器，并提出一种新方法DeepTopPush，用于最小化顶部损失函数。由于阈值依赖于所有样本，该问题不可分解。我们修改了随机梯度下降法，以端到端训练方式处理不可分解性，并提出一种仅基于当前小批量值和一个延迟值估计阈值的方法。我们在视觉识别数据集和两个实际应用中展示了DeepTopPush的优异性能。第一个应用是选择少量分子用于进一步药物测试。第二个应用使用真实恶意软件数据，在极低的虚警率$10^{-5}$下检测到了46%的恶意软件。