CliqueParcel: An Approach For Batching LLM Prompts That Jointly Optimizes Efficiency And Faithfulness

Understand

Large language models (LLMs) have become pivotal in recent research.

• However, during the inference process, LLMs still require substantial resources.
• In this paper, we propose CliqueParcel, a method designed to improve the efficiency of LLMs via prompt batching.
• Existing strategies to optimize inference efficiency often compromise on output quality, leading to a discounted output problem.