We provide algorithms for regression with adversarial responses under large classes of non-i.i.d. instance sequences, on general separable metric spaces, with provably minimal assumptions. We also give characterizations of learnability in this regression context. We consider universal consistency which asks for strong consistency of a learner without restrictions on the value responses. Our analysis shows that such an objective is achievable for a significantly larger class of instance sequences than stationary processes, and unveils a fundamental dichotomy between value spaces: whether finite-horizon mean estimation is achievable or not. We further provide optimistically universal learning rules, i.e., such that if they fail to achieve universal consistency, any other algorithms will fail as well. For unbounded losses, we propose a mild integrability condition under which there exist algorithms for adversarial regression under large classes of non-i.i.d. instance sequences. In addition, our analysis also provides a learning rule for mean estimation in general metric spaces that is consistent under adversarial responses without any moment conditions on the sequence, a result of independent interest.

翻译：我们提出了在广义可分离度量空间上，针对大规模非独立同分布实例序列的对抗响应回归算法，其假设条件在理论上达到最小化。同时，我们给出了该回归场景中可学习性的刻画。我们考虑通用一致性，即要求学习器在无需限制响应值的情况下具有强一致性。分析表明，该目标对于远大于平稳过程的一类实例序列均可实现，并揭示了值空间中的基本二分性：有限域均值估计是否可实现。我们进一步提供了乐观通用学习规则，即若该类规则无法实现通用一致性，则其他任何算法也将失效。对于无界损失，我们提出一个温和的可积性条件，在该条件下存在适用于大规模非独立同分布实例序列的对抗回归算法。此外，我们的分析还给出了广义度量空间中均值估计的学习规则，该规则在无序列矩条件限制下对对抗响应仍保持一致性，这一结果具有独立研究价值。