Statistical hypothesis testing and effect size measurement are routine parts of quantitative research. Advancements in computer processing power have greatly improved the capability of statistical inference through the availability of resampling methods. However, many of the statistical practices used today are based on traditional, parametric methods that rely on assumptions about the underlying population. These assumptions may not always be valid, leading to inaccurate results and misleading interpretations. Permutation testing, on the other hand, generates the sampling distribution empirically by permuting the observed data, providing distribution-free hypothesis testing. Furthermore, this approach lends itself to a powerful method for multiple comparison correction - known as max correction - which is less prone to type II errors than conventional correction methods. Parametric methods have also traditionally been utilized for estimating the confidence interval of various test statistics and effect size measures. However, these too can be estimated empirically using permutation or bootstrapping techniques. Whilst resampling methods are generally considered preferable, many popular programming languages and statistical software packages lack efficient implementations. Here, we introduce PERMUTOOLS, a MATLAB package for multivariate permutation testing and effect size measurement.

翻译：统计假设检验和效应量测量是定量研究的常规组成部分。计算机处理能力的提升通过重采样方法的普及，极大地增强了统计推断的能力。然而，当今使用的许多统计实践仍基于传统的参数方法，这些方法依赖于对总体分布的假设。这些假设并非总是成立，可能导致不准确的结果和误导性解释。相比之下，置换检验通过对观测数据进行置换来经验性地生成抽样分布，提供了一种无分布假设的检验方法。此外，这种方法还衍生出一种强大的多重比较校正方法——即最大值校正——该方法相比传统校正方法更不易出现第二类错误。传统上，参数方法也被用于估计各种检验统计量和效应量度量的置信区间。然而，这些区间同样可以使用置换或自助法技术进行经验性估计。尽管重采样方法通常被认为更优，但许多流行的编程语言和统计软件包缺乏高效的实现。在此，我们介绍PERMUTOOLS，一个用于多元置换检验和效应量测量的MATLAB工具包。