This research's primary motivation of this study is to address the high hardware and computational demands typically associated with LLMs.Therefore,our goal is to find a balance between model lightness and performance,striving to maximize performance while using a comparatively lightweight model. Hyacinth6B was developed with this objective in mind,aiming to fully leverage the core capabilities of LLMs without incurring substantial resource costs, effectively pushing the boundaries of smaller model's performance. The training approach involves parameter efficient finetuning using the LoRA method.
翻译:本研究的主要動機在於解決大型語言模型通常伴隨的高硬體與計算需求。因此,我們致力於在模型輕量化與效能之間取得平衡,力求在使用相對輕量模型的同時最大化其表現。Hyacinth6B 即為此目標而開發,旨在充分利用大型語言模型的核心能力,且無需投入大量資源,有效突破較小模型的效能極限。訓練方法採用基於 LoRA 的參數高效微調技術。