XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameters

Understand

In recent years, pre-trained language models have undergone rapid development with the emergence of large-scale models.

• However, there is a lack of open-sourced chat models specifically designed for the Chinese language, especially in the field of Chinese finance, at the scale of hundreds of billions.
• To address this gap, we introduce XuanYuan 2.0, the largest Chinese chat model to date, built upon the BLOOM-176B architecture.
• Additionally, we propose a novel training method called hybrid-tuning to mitigate catastrophic forgetting.