Large language models (LLMs) have garnered significant attention, but the definition of "large" lacks clarity. This paper focuses on medium-sized language models (MLMs), defined as having at least six billion parameters but less than 100 billion. The study evaluates MLMs regarding zero-shot generative question answering, which requires models to provide elaborate answers without external document retrieval. The paper introduces an own test dataset and presents results from human evaluation. Results show that combining the best answers from different MLMs yielded an overall correct answer rate of 82.7% which is better than the 60.9% of ChatGPT. The best MLM achieved 71.8% and has 33B parameters, which highlights the importance of using appropriate training data for fine-tuning rather than solely relying on the number of parameters. More fine-grained feedback should be used to further improve the quality of answers. The open source community is quickly closing the gap to the best commercial models.

翻译：大型语言模型（LLMs）引起了广泛关注，但“大型”这一概念缺乏明确界定。本文聚焦于中等规模语言模型（MLMs），将其定义为参数规模介于60亿至1000亿之间的模型。研究评估了MLMs在零样本生成式问答任务中的表现，该任务要求模型无需外部文档检索即可提供详尽答案。论文引入了自主测试数据集，并展示了人工评估结果。结果表明，整合不同MLM的最佳答案后，总体正确答案率达到82.7%，优于ChatGPT的60.9%。表现最佳的MLM（参数规模330亿）正确率为71.8%，凸显了使用恰当训练数据进行微调的重要性，而非单纯依赖参数数量。未来应引入更细粒度的反馈机制以进一步提升回答质量。开源社区正在迅速缩小与最优商业模型之间的差距。