Korean folk painting (minhwa) is built from a small vocabulary of auspicious symbols, a tiger for protection, a pair of birds for marital harmony, a peony for wealth, that recur across many of its painted genres. This suggests an obvious computational approach, identify which symbols appear in a painting and read the genre from the inventory. Working with a public corpus that pairs whole paintings, eight-field bilingual curatorial captions, and a separate set of expert object crops, we find that this approach does not work. A model given only a list of which symbols a painting contains predicts the genre far worse than a model that fuses the image with the curatorial text, and forcing the genre representation to be object-grounded actively hurts accuracy. The visual evidence on which the genre prediction rests is nonetheless localized and inspectable. A leakage-safe object evidence map projected from a part-level detector is spatially faithful to where curators isolated symbolic objects and to a patch-based surrogate's own gradient saliency. We name this configuration a faithful-but-insufficient dissociation. The part-level explanation is honest about what the part-level model sees, yet the genre target turns on how symbols are arranged rather than on which ones appear. The same lens separates a content label that survives transfer to held-out source institutions, genre, from a style label that does not, era, a prediction we confirm on two further labels in the corpus. We release the multimodal system, a worked-example reading of one painting's evidence map against its catalogue, and a set of evaluation cautions that recur in long-tailed heritage collections.
翻译:韩国民俗画(minhwa)由少量吉祥符号构成——老虎象征保护、双鸟象征婚姻和谐、牡丹象征富贵——这些符号在其众多绘画类型中反复出现。这提示了一种直观的计算方法:识别画作中出现的符号并从符号清单中解读类型。基于一个公开语料库(包含完整画作、八字段双语策展说明文字及一组独立的专家对象裁剪图),我们发现此方法难以奏效。仅凭画作所含符号列表进行类型预测的模型,其准确率远低于将图像与策展文字融合的模型;而强制类型表征基于对象定位反而会损害准确性。尽管如此,支撑类型预测的视觉证据仍具有局部性与可检验性:从部件级检测器投影出的防泄漏对象证据图,在空间上忠实于策展人隔离象征对象的位置,以及基于块的代理模型自身的梯度显著性。我们将此配置命名为"忠实但不足的分离":部件级解释诚实地反映了部件级模型的观察,但类型预测目标取决于符号的排布方式而非出现与否。同一视角区分出两类标签——可在跨源机构迁移的内容标签(类型)与不可迁移的风格标签(时代)——我们通过语料库中另外两类标签验证了这一预测。我们开源了多模态系统、一幅画作证据图与馆藏目录对照的实例解读,以及针对长尾遗产语料库反复出现的评估注意事项。