Experimental designs reflect assumptions about variable relationships that determine what causal queries researchers can answer through the experiment. Accounting for and communicating these assumptions is essential for drawing valid, generalizable conclusions from scientific experiments. Unfortunately, existing experimental design tools elide these details, expecting researchers to reason about design decisions and assumptions on their own. To surface assumptions and enable design exploration, we introduce a grammar of composable operators for constructing experimental assignment procedures grounded in matrix algebra. The PLanet DSL implements this grammar and compiles PLanet programs into constraint satisfaction problems over matrices. Together, PLanet's composable grammar and matrix representation enable a static analysis to determine which causal queries are testable under different assumptions. In an expressivity evaluation, PLanet was the most expressive of existing DSLs. Critical reflections with the authors of these DSLs revealed that PLanet makes design choices explicit without requiring procedural specification. Think-aloud studies showed that PLanet facilitated design exploration and surfaced assumptions researchers may otherwise overlook.
翻译:实验设计反映了关于变量关系的假设,这些假设决定了研究者通过实验能够回答哪些因果查询。阐明并沟通这些假设对于从科学实验中得出有效且可泛化的结论至关重要。然而,现有的实验设计工具忽略了这些细节,期望研究者自行推敲设计决策与假设。为显化假设并支持设计探索,我们引入了一种基于矩阵代数的可组合算子语法,用于构建实验分配程序。PLanet领域特定语言实现了该语法,并将PLanet程序编译为矩阵上的约束满足问题。通过将可组合语法与矩阵表示相结合,PLanet能够进行静态分析,从而判定在不同假设下哪些因果查询是可检验的。在表达性评估中,PLanet是现有领域特定语言中表达能力最强的。与这些领域特定语言作者的批判性反思表明,PLanet无需过程式规范即可明确设计选择。Think-aloud研究显示,PLanet促进了设计探索,并能使研究者注意到原本可能忽略的假设。