Bone Soups: A Seek-and-Soup Model Merging Approach for Controllable Multi-Objective Generation

Understand

User information needs are often highly diverse and varied.

• A key challenge in current research is how to achieve controllable multi-objective generation while enabling rapid adaptation to accommodate diverse user demands during test time.
• Existing solutions, such as Rewarded Soup, focus on merging language models individually tuned on single objectives.
• While easy to implement and widely used, these approaches face limitations in achieving optimal performance due to their disregard for the impacts of competing objectives on model tuning.