Knowledge of syntax includes knowledge of rare, idiosyncratic constructions. LLMs must overcome frequency biases in order to master such constructions. In this study, I prompt GPT-3 to give acceptability judgments on the English-language Article + Adjective + Numeral + Noun construction (e.g., "a lovely five days"). I validate the prompt using the CoLA corpus of acceptability judgments and then zero in on the AANN construction. I compare GPT- 3's judgments to crowdsourced human judgments on a subset of sentences. GPT-3's judgments are broadly similar to human judgments and generally align with proposed constraints in the literature but, in some cases, GPT-3's judgments and human judgments diverge from the literature and from each other.
翻译:句法知识包含对罕见、特殊构式的认知。大语言模型需克服频率偏差才能掌握此类构式。本研究通过提示GPT-3对英语“冠词+形容词+数词+名词”构式(如“a lovely five days”)输出可接受性判断。首先使用CoLA可接受性判断语料库验证提示有效性,继而聚焦AANN构式。将GPT-3的判断与基于众包的人类判断进行子集句子对比。结果显示,GPT-3的判断与人类判断整体相似,且大体符合文献提出的约束条件,但在某些案例中GPT-3的判断与人类判断既偏离文献所述,彼此间亦存在差异。