Imagine conducting a study to determine whether users rate the visual aesthetics of your website more positively than your competitors. To assess users' perceptions of both websites, you use a validated survey scale for visual aesthetics, and you observe a statistically significant difference in users' ratings of the visual aesthetics of the two websites of 0.5 on a 7-point Likert-type scale. However, determining whether such a difference is practically (and theoretically) meaningful is challenging. In this paper, I follow the procedure outlined in Anvari & Lakens (2021) to determine the smallest subjectively experienced difference in VisAWI-s ratings using an anchor-based method. A sample of N = 249 participants rated and compared screenshots of eight websites in an online survey. I determined an estimate of a population-specific mean difference of 0.4, or in POMP units 6.58%, which translates to a mean difference of 0.46 with the 7-point Likert-type scale of the VisAWI-s. These values suggest that differences in VisAWI-s scores exceeding these estimates, such as the 0.5 mentioned above, are likely noticeable and meaningful to users. However, the estimate of this smallest subjectively experienced difference is affected by the overall visual aesthetics rating of the stimuli used. Researchers can use this effect size to inform study design and sample size planning. Still, whenever possible, they should aim to determine a domain- and research-design-specific smallest effect size of interest.

翻译：设想您正在进行一项研究，旨在探讨用户对您网站视觉美学的评价是否优于竞争对手。为评估用户对两类网站的感知，您采用了经过验证的视觉美学调查量表，并观察到两者在7点李克特量表上的视觉美学评分差异具有统计学意义（差值为0.5）。然而，判定此类差异是否具有实践（与理论）意义颇具挑战性。本文遵循Anvari与Lakens（2021）提出的锚定法流程，通过锚定方法确定VisAWI-s评分中主观感知的最小差异。研究通过在线调查招募249名被试，对八个网站截图的视觉美学进行评分与比较。最终确定总体特异性平均差估计值为0.4（以POMP单位表示为6.58%），对应VisAWI-s量表7点李克特评分中的0.46平均差。该数值表明：超出此估计值（如前文所述0.5）的VisAWI-s评分差异更可能被用户感知并具有实际意义。需注意，该主观感知最小差异估计值受刺激材料整体视觉美学评分水平的影响。研究者可将该效应量用于研究设计与样本量规划，但应尽可能确定特定领域与研究设计的最小感兴趣效应量。