Recent work on predicting category structure with distributional models, using either static word embeddings (Heyman and Heyman, 2019) or contextualized language models (CLMs) (Misra et al., 2021), report low correlations with human ratings, thus calling into question their plausibility as models of human semantic memory. In this work, we revisit this question testing a wider array of methods for probing CLMs for predicting typicality scores. Our experiments, using BERT (Devlin et al., 2018), show the importance of using the right type of CLM probes, as our best BERT-based typicality prediction methods substantially improve over previous works. Second, our results highlight the importance of polysemy in this task: our best results are obtained when using a disambiguation mechanism. Finally, additional experiments reveal that Information Contentbased WordNet (Miller, 1995), also endowed with disambiguation, match the performance of the best BERT-based method, and in fact capture complementary information, which can be combined with BERT to achieve enhanced typicality predictions.

翻译：近期，基于分布语义模型（使用静态词向量（Heyman 和 Heyman, 2019）或上下文语言模型（Misra 等, 2021））预测类别结构的研究报告指出，其与人类评分之间的相关性较低，从而质疑了这些模型作为人类语义记忆模型的合理性。本研究重新审视该问题，测试了更多样化的方法，从上下文语言模型中提取信息以预测典型性评分。基于 BERT（Devlin 等, 2018）的实验表明，采用适当的上下文语言模型探测方式至关重要——我们最优的基于 BERT 的典型性预测方法较以往研究有显著提升。其次，研究结果凸显了多义词在此任务中的重要性：采用消歧机制时取得了最优结果。最后，附加实验揭示：具备消歧能力的基于信息含量的 WordNet（Miller, 1995）在性能上与最优 BERT 方法持平，且实际上捕获了互补信息，可与 BERT 结合实现更优的典型性预测。