Hidden in Plain Sight: Exploring Chat History Tampering in Interactive Language Models

Understand

Large Language Models (LLMs) such as ChatGPT and Llama have become prevalent in real-world applications, exhibiting impressive text generation performance.

• LLMs are fundamentally developed from a scenario where the input data remains static and unstructured.
• To behave interactively, LLM-based chat systems must integrate prior chat history as context into their inputs, following a pre-defined structure.
• However, LLMs cannot separate user inputs from context, enabling chat history tampering.