Fetching the paper…

Reading the bibliography…

2023

Baselines for Identifying Watermarked Large Language Models

Tang, Leonard, Uberti, Gavin, Shlomi, Tom

Original
PDF
Post on X

Understand

We consider the emerging problem of identifying the presence and use of watermarking schemes in widely used, publicly hosted, closed source large language models (LLMs).

We introduce a suite of baseline algorithms for identifying watermarks in LLMs that rely on analyzing distributions of output tokens and logits generated by watermarked and unmarked LLMs.
Notably, watermarked LLMs tend to produce distributions that diverge qualitatively and identifiably from standard models.
Furthermore, we investigate the identifiability of watermarks at varying strengths and consider the tradeoffs of each of our identification mechanisms with respect to watermarking scenario.

Baselines for Identifying Watermarked Large Language Models · Around

Built on

The dip test of unimodality
Hartigan, J. A. and Hartigan, P. M · 1985
Earlier work this paper cites.
Crafting papers on machine learning
Langley, P · 2000
Earlier work this paper cites.
Natural language watermarking
Topkara, M., Taskiran, C. M., and Delp III, E. J · 2005
Earlier work this paper cites.
Release strategies and the social impacts of language models
Original
Solaiman, I., Brundage, M., Clark, J., Askell, A., Herbert-Voss, A., Wu, J., Radford, A., Krueger, G., Kim, J. W., Kreps, S., et al · 2019
Earlier work this paper cites.

Similar

My AI safety projects at OpenAI
Aaronson, S · 2023
Cited alongside, same era.
How close is chatgpt to human experts? comparison corpus, evaluation, and detection
Original
Guo, B., Zhang, X., Wang, Z., Jiang, M., Nie, J., Ding, Y., Yue, J., and Wu, Y · 2023
Cited alongside, same era.
A watermark for large language models
Original
Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T · 2023
Cited alongside, same era.

Then

Detectgpt: Zero-shot machine-generated text detection using probability curvature
Original
Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., and Finn, C · 2023
Closest in time.
New AI classifier for indicating AI-written text
OpenAI · 2023
Closest in time.
Can ai-generated text be reliably detected?
Original
Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., and Feizi, S · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…