This research's primary motivation of this study is to address the high hardware and computational demands typically associated with LLMs.Therefore,our goal is to find a balance between model lightness and performance,striving to maximize performance while using a comparatively lightweight model. Hyacinth6B was developed with this objective in mind,aiming to fully leverage the core capabilities of LLMs without incurring substantial resource costs, effectively pushing the boundaries of smaller model's performance. The training approach involves parameter efficient finetuning using the LoRA method.
翻译:本研究的主要动机在于解决大型语言模型通常伴随的高硬件与计算需求问题。因此,我们的目标是平衡模型轻量性与性能,力求在采用相对轻量级模型的同时最大化性能。Hyacinth6B正是基于这一目标开发,旨在无需承担大量资源成本的情况下,充分释放大型语言模型的核心能力,从而有效突破较小模型的表现边界。训练方法采用基于LoRA的参数高效微调技术。