Deep learning models can generate virtual immunohistochemistry (IHC) stains from hematoxylin and eosin (H&E) images, offering a scalable and low-cost alternative to laboratory IHC. However, reliable evaluation of image quality remains a challenge as current texture- and distribution-based metrics quantify image fidelity rather than the accuracy of IHC staining. Here, we introduce an automated and accuracy grounded framework to determine image quality across sixteen paired or unpaired image translation models. Using color deconvolution, we generate masks of pixels stained brown (i.e., IHC-positive) as predicted by each virtual IHC model. We use the segmented masks of real and virtual IHC to compute stain accuracy metrics (Dice, IoU, Hausdorff distance) that directly quantify correct pixel - level labeling without needing expert manual annotations. Our results demonstrate that conventional image fidelity metrics, including Frechet Inception Distance (FID), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM), correlate poorly with stain accuracy and pathologist assessment. Paired models such as PyramidPix2Pix and AdaptiveNCE achieve the highest stain accuracy, whereas unpaired diffusion- and GAN-based models are less reliable in providing accurate IHC positive pixel labels. Moreover, whole-slide images (WSI) reveal performance declines that are invisible in patch-based evaluations, emphasizing the need for WSI-level benchmarks. Together, this framework defines a reproducible approach for assessing the quality of virtual IHC models, a critical step to accelerate translation towards routine use by pathologists.

翻译：深度学习模型能够从苏木精-伊红（H&E）图像生成虚拟免疫组化（IHC）染色，为实验室IHC提供了一种可扩展且低成本的替代方案。然而，图像质量的可靠评估仍然是一个挑战，因为当前基于纹理和分布的指标量化的是图像保真度而非IHC染色的准确性。本文提出了一种基于准确性的自动化框架，用于评估十六种配对或非配对图像转换模型的图像质量。通过颜色解卷积，我们生成每个虚拟IHC模型预测的棕色染色像素（即IHC阳性）掩码。利用真实与虚拟IHC的分割掩码，我们计算染色准确度指标（Dice系数、交并比、豪斯多夫距离），这些指标直接量化像素级标记的正确性，无需专家手动标注。我们的结果表明，传统图像保真度指标，包括弗雷歇起始距离（FID）、峰值信噪比（PSNR）和结构相似性（SSIM），与染色准确度及病理学家评估的相关性较弱。配对模型如PyramidPix2Pix和AdaptiveNCE实现了最高的染色准确度，而非配对的基于扩散和生成对抗网络的模型在提供准确的IHC阳性像素标记方面可靠性较低。此外，全玻片图像（WSI）揭示了在基于图像块的评估中无法察觉的性能下降，强调了WSI级基准测试的必要性。综上，该框架定义了一种可重复的方法来评估虚拟IHC模型的质量，这是加速其向病理学家常规应用转化的关键一步。