Machine Learning (ML) is an expressive framework for turning data into computer programs. Across many problem domains -- both in industry and policy settings -- the types of computer programs needed for accurate prediction or optimal control are difficult to write by hand. On the other hand, collecting instances of desired system behavior may be relatively more feasible. This makes ML broadly appealing, but also induces data sensitivities that often manifest as unexpected failure modes during deployment. In this sense, the training data available tend to be imperfect for the task at hand. This thesis explores several data sensitivities of modern machine learning and how to address them. We begin by discussing how to prevent ML from codifying prior human discrimination measured in the training data, where we take a fair representation learning approach. We then discuss the problem of learning from data containing spurious features, which provide predictive fidelity during training but are unreliable upon deployment. Here we observe that insofar as standard training methods tend to learn such features, this propensity can be leveraged to search for partitions of training data that expose this inconsistency, ultimately promoting learning algorithms invariant to spurious features. Finally, we turn our attention to reinforcement learning from data with insufficient coverage over all possible states and actions. To address the coverage issue, we discuss how causal priors can be used to model the single-step dynamics of the setting where data are collected. This enables a new type of data augmentation where observed trajectories are stitched together to produce new but plausible counterfactual trajectories.

翻译：机器学习是一种将数据转化为计算机程序的表达性框架。在许多问题领域——无论是工业界还是政策制定场景——要实现准确的预测或最优控制所需的计算机程序往往难以手动编写。另一方面，收集期望系统行为的实例则相对更为可行。这使得机器学习具有广泛吸引力，但也带来了数据敏感性问题，这些敏感问题通常在部署阶段表现为意外的故障模式。从这个意义上说，可用的训练数据往往不适合当前任务。本论文探讨了现代机器学习中的若干数据敏感问题及其应对方法。我们首先讨论如何防止机器学习固化训练数据中测量到的先前人类歧视，对此我们采用公平表示学习方法。接着讨论如何从包含虚假特征的数据中学习，这些特征在训练时能提供预测准确性，但在部署时却不可靠。我们观察到，当标准训练方法倾向于学习此类特征时，可以利用这种倾向来搜索暴露这种不一致性的训练数据分区，最终促进学习算法对虚假特征保持不变性。最后，我们将注意力转向从覆盖不充分的状态和动作数据中进行强化学习。为解决覆盖问题，我们讨论了如何利用因果先验来建模数据收集环境的单步动力学特性。这使得一种新型数据增强方法成为可能，即将观测到的轨迹拼接起来，生成新的但合理的反事实轨迹。