Balancing Exploration and Exploitation in LLM using Soft RLLF for Enhanced Negation Understanding

Understand

Finetuning approaches in NLP often focus on exploitation rather than exploration, which may lead to suboptimal models.

• Given the vast search space of natural language, this limited exploration can restrict their performance in complex, high-stakes domains, where accurate negation understanding and logical reasoning abilities are crucial.
• To address this issue, we leverage Reinforcement Learning from Logical Feedback (RLLF) to create an effective balance between exploration and exploitation in LLMs.
• Our approach employs an appropriate benchmark dataset for training and evaluation, highlighting the importance of exploration in enhancing negation understanding capabilities.