Visual validation of regression models in scatterplots is a common practice for assessing model quality, yet its efficacy remains unquantified. We conducted two empirical experiments to investigate individuals' ability to visually validate linear regression models (linear trends) and to examine the impact of common visualization designs on validation quality. The first experiment showed that the level of accuracy for visual estimation of slope (i.e., fitting a line to data) is higher than for visual validation of slope (i.e., accepting a shown line). Notably, we found bias toward slopes that are "too steep" in both cases. This lead to novel insights that participants naturally assessed regression with orthogonal distances between the points and the line (i.e., ODR regression) rather than the common vertical distances (OLS regression). In the second experiment, we investigated whether incorporating common designs for regression visualization (error lines, bounding boxes, and confidence intervals) would improve visual validation. Even though error lines reduced validation bias, results failed to show the desired improvements in accuracy for any design. Overall, our findings suggest caution in using visual model validation for linear trends in scatterplots.

翻译：散点图中回归模型的视觉验证是评估模型质量的常见做法，但其有效性尚未得到量化。我们通过两项实证实验研究了人们视觉验证线性回归模型（线性趋势）的能力，并检验了常见可视化设计对验证质量的影响。首项实验表明，斜率视觉估计（即对数据拟合直线）的准确度高于斜率视觉验证（即接受给定直线）。值得注意的是，我们发现两种情况下均存在对"过陡"斜率的系统性偏差。这揭示了新颖的发现：参与者本能地采用点与直线之间的正交距离（即ODR回归）进行评估，而非常见的垂直距离（OLS回归）。在第二项实验中，我们探究了引入回归可视化常用设计元素（误差线、边界框和置信区间）是否能改善视觉验证效果。尽管误差线降低了验证偏差，但所有设计均未实现预期中的准确度提升。总体而言，我们的研究结果表明，在散点图中对线性趋势进行视觉模型验证需保持审慎态度。