Watermarking LLMs with Weight Quantization

Understand

Abuse of large language models reveals high risks as large language models are being deployed at an astonishing speed.

• It is important to protect the model weights to avoid malicious usage that violates licenses of open-source large language models.
• This paper proposes a novel watermarking strategy that plants watermarks in the quantization process of large language models without pre-defined triggers during inference.
• The watermark works when the model is used in the fp32 mode and remains hidden when the model is quantized to int8, in this way, the users can only inference the model without further supervised fine-tuning of the model.