Teaching Specific Scientific Knowledge into Large Language Models through Additional Training

Understand

Through additional training, we explore embedding specialized scientific knowledge into the Llama 2 Large Language Model (LLM).

• Key findings reveal that effective knowledge integration requires reading texts from multiple perspectives, especially in instructional formats.
• We utilize text augmentation to tackle the scarcity of specialized texts, including style conversions and translations.
• Hyperparameter optimization proves crucial, with different size models (7b, 13b, and 70b) reasonably undergoing additional training.