Counterfactual explanations are widely used to communicate how inputs must change for a model to alter its prediction. For a single instance, many valid counterfactuals can exist, which leaves open the possibility for an explanation provider to cherry-pick explanations that better suit a narrative of their choice, highlighting favourable behaviour and withholding examples that reveal problematic behaviour. We formally define cherry-picking for counterfactual explanations in terms of an admissible explanation space, specified by the generation procedure, and a utility function. We then study to what extent an external auditor can detect such manipulation. Considering three levels of access to the explanation process: full procedural access, partial procedural access, and explanation-only access, we show that detection is extremely limited in practice. Even with full procedural access, cherry-picked explanations can remain difficult to distinguish from non cherry-picked explanations, because the multiplicity of valid counterfactuals and flexibility in the explanation specification provide sufficient degrees of freedom to mask deliberate selection. Empirically, we demonstrate that this variability often exceeds the effect of cherry-picking on standard counterfactual quality metrics such as proximity, plausibility, and sparsity, making cherry-picked explanations statistically indistinguishable from baseline explanations. We argue that safeguards should therefore prioritise reproducibility, standardisation, and procedural constraints over post-hoc detection, and we provide recommendations for algorithm developers, explanation providers, and auditors.

翻译：反事实解释被广泛用于说明输入需如何改变才能促使模型改变其预测结果。针对单个实例，可能存在多种有效的反事实解释，这为解释提供者留下了筛选性选择的空间——他们可能刻意选取更符合自身预设叙事的解释，突出有利行为而隐藏揭示问题行为的示例。我们基于可容许解释空间（由生成过程定义）和效用函数，正式定义了反事实解释中的筛选性选择。随后研究了外部审计者在何种程度上能够检测此类操纵行为。通过考察对解释过程的三种访问权限：完整程序访问、部分程序访问和仅解释访问，我们发现实际检测能力极为有限。即使拥有完整程序访问权限，筛选性选择的解释仍可能难以与非筛选性选择的解释区分，因为有效反事实的多重性及解释规范的灵活性为掩盖刻意选择提供了足够的自由度。实证研究表明，这种变异性通常超过筛选性选择对标准反事实质量指标（如邻近性、合理性和稀疏性）的影响，使得筛选性选择的解释在统计上与基线解释无法区分。我们认为，保障措施应优先考虑可重复性、标准化和程序约束，而非事后检测，并为此向算法开发者、解释提供者和审计者提出了具体建议。