Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost

Understand

Today's large language models (LLMs) can solve challenging question-answering tasks, and prompt engineering techniques, such as chain-of-thought (CoT), have gained attention for enhancing the explanation and correctness of outputs.

• However, many models and techniques tend to produce excessively verbose and lengthy answers, leading to issues with both conciseness and generation time.
• To address this, this paper analyzes the impact of output lengths on LLM inference pipelines by introducing and proposing novel metrics to evaluate the \textit{correct conciseness} of a model and related prompting techniques.
• Then, we examine the impact of controlling output length through a refined prompt engineering strategy, Constrained-CoT (CCoT), which encourages the model to produce more concise outputs.