SciDFM: A Large Language Model with Mixture-of-Experts for Science

Understand

Recently, there has been a significant upsurge of interest in leveraging large language models (LLMs) to assist scientific discovery.

• However, most LLMs only focus on general science, while they lack domain-specific knowledge, such as chemical molecules and amino acid sequences.
• To bridge these gaps, we introduce SciDFM, a mixture-of-experts LLM, which is trained from scratch and is able to conduct college-level scientific reasoning and understand molecules and amino acid sequences.
• We collect a large-scale training corpus containing numerous scientific papers and books from different disciplines as well as data from domain-specific databases.