In Semantic-ID (SID) based generative recommendation, each item is represented as a sequence of discrete codes, and an autoregressive model is trained to generate the SID sequence of the next item; top-K performance is then measured by checking whether the SID sequence of the target item appears among the generated sequences. This evaluation protocol equates SID-level matching with item-level recommendation, an equivalence that holds only when every SID sequence maps to a single item. We show this assumption breaks down in practice: because tokenizers compress item features into a code space, semantically similar but collaboratively distinct items are frequently assigned the same SID sequence. Across four datasets and five representative tokenizers, the fraction of items involved in such collisions reaches 30.5%, so matching a shared SID sequence identifies only a collision group rather than the target item. Consequently, SID-level metrics overestimate item-level performance (Hit@10 is inflated by up to 103.36%), and the inflation grows with the collision rate. To support faithful comparison, we develop collision-aware item-level metrics computed directly from generated SID sequences, together with a post-tokenizer procedure that reassigns last-level SIDs at minimum cost to obtain a collision-free assignment for any existing tokenizer. Our results indicate that SID-level rankings in prior work should be interpreted with caution, and that reliable tokenizer evaluation requires either item-level correction or collision-free SID assignments.

翻译：在基于语义ID（SID）的生成式推荐中，每个物品被表示为一系列离散编码，并通过训练自回归模型生成下一物品的SID序列；随后通过检查目标物品的SID序列是否出现在生成的序列中来衡量Top-K性能。该评估协议将SID级别的匹配等同于物品级别的推荐，这种等价性仅在每个SID序列唯一映射到单个物品时才成立。我们证明这一假设在实际中不成立：由于分词器将物品特征压缩到编码空间，语义相似但在协同过滤中不同的物品经常被分配相同的SID序列。在四个数据集和五个代表性分词器上，涉及此类冲突的物品比例高达30.5%，因此匹配共享的SID序列仅能识别冲突组而非目标物品。结果，SID级别的指标高估了物品级别的性能（Hit@10最多虚增103.36%），且虚增程度随冲突率增加而增大。为支持可信的比较，我们开发了直接从生成的SID序列计算的冲突感知物品级指标，以及一种后分词器处理程序：以最小代价重新分配最后一层SID，从而为任何现有分词器获得无冲突分配。我们的结果表明，先前工作中基于SID级别的排名应谨慎解读，而可靠的分词器评估需要物品级修正或无冲突的SID分配。