We present ImageReward -- the first general-purpose text-to-image human preference reward model -- to address various prevalent issues in generative models and align them with human values and preferences. Its training is based on our systematic annotation pipeline that covers both the rating and ranking components, collecting a dataset of 137k expert comparisons to date. In human evaluation, ImageReward outperforms existing scoring methods (e.g., CLIP by 38.6\%), making it a promising automatic metric for evaluating and improving text-to-image synthesis. The reward model is publicly available via the \texttt{image-reward} package at \url{https://github.com/THUDM/ImageReward}.

翻译：我们提出ImageReward——首个通用型文本到图像人类偏好奖励模型——以应对生成模型中普遍存在的各类问题，并将其与人类价值观和偏好对齐。该模型的训练基于我们系统化的标注流程，涵盖评分与排序两个环节，目前已完成包含13.7万组专家比较的数据集构建。在人工评估中，ImageReward优于现有评分方法（例如，较CLIP提升38.6%），使其成为评估和改善文本到图像合成任务的有效自动化指标。该奖励模型已通过\texttt{image-reward}软件包在\url{https://github.com/THUDM/ImageReward}页面公开提供。